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Preface 



The National Evaluation of the Even Start Family Literacy Program was a four-year 
national effort designed to describe the types of Even Start projects that were funded, the 
services provided, the collaborative efforts undertaken, and the obstacles to program 
implementation that were encountered. The evaluation also describes the families that 
participated in Even Start, the services they received, and the effects of Even Start 
participation on children's school readiness; parent's literacy, parenting, and personal 
skills; and family stability and resources. Finally, the evaluation provided assistance to 
Even Start projects to conduct locally-designed evaluations, and to prepare and submit 
applications to the Department of Education for entry into the National Diffusion Network. 

This is the final report from the National Even Start Evaluation. It provides descriptive 
information about the first four cohorts of Even Start projects (340 projects first funded 
in the fall of 1989, 1990, 1991 or 1992), as well as findings about program impacts 
based on data from the first two cohorts of projects (1 20 projects first funded in the fall 
of 1989 or 1990). 
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To be eligible for Even Start, a family must have an adult who is eligible to participate in 
an adult education program under the Adult Education Act, and who is a parent of a child 
under than eight years of age who lives in a Chapter 1 elementary school attendance 
area. Even Start projects must provide participating families with an integrated program 
of early childhood education, adult basic skills training, and parenting education. The 
program's design is based on the notion that these components build on each other and 
that families need to receive all three services, not just one or two, in order to effect 
lasting change and improve children's school success. As a "family-focused" rather than 
parent- or child-focused program, Even Start has three interrelated goals: 

• to help parents become full partners in the education of their children, 

• to assist children in reaching their full potential as learners, and 

• to provide literacy training for their parents. 

Even Start began as a federal demonstration program administered by the U.S. 
Department of Education (ED) that provided school districts with four-year discretionary 
grants for family literacy projects. According to the Even Start statute, when the program 
is funded for $50 million or more per year, it must be administered at the state level. 
Thus, in 1992, most grant-making responsibilities were delegated to the states although 
the federal government remained responsible for grants for Migrant Education projects and 
grants to Indian tribes and tribal organizations. Each state's share of Even Start funds is 
based on its proportion of Chapter 1 Basic Grants funds. States hold grant competitions 
and make subgrant awards. The statute specifies that each Even Start subgrantee must 
receive a minimum of $75,000 per year. Exhibit 1.1 summarizes Even Start's funding 
history and Exhibit 1 .2 shows the location of Even Start projects first funded in the fall 
of 1989, 1990, 1991, or 1992. 





Exhibit 1.1 






Even Start Funding History 




Fiscal Year 


Federal Funding 


Number of Projects 


1989 


$14,820,000 


76 


1990 


$24,201,000 


123 


1991 


$49,770,000 


234 


1992 


$70,000,000 


340 


1993 


$89,123,000 


440 


1994 


$91,373,000 


450 


Exhibit reads: In fiscal year 1991, $49,8 million of federal funding was used to provide Even Start 
grants to 234 local projects. 


Source: U.S. Department of Education, Office of Policy and Planning. 
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Exhibit 1 .2 

Location of Even Start Projects 
(Projects First Funded in 1989. 1990, 1991, and 1992) 
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Source: U.S. Department of Education, Office of Policy and Planning, April 1994. 



Mandate for the Evaluation 



Section 1058 of the Even Start legislation required an independent national evaluation of 
the projects funded under Even Start. This section reads as follows: 

(a) Independent Annual Evaluation . The Secretary shall provide for the annual 
independent evaluation of programs under this part to determine their 
effectiveness in providing: 

(V services to special populations; 

(2) adult education services; 

(3) parent training; 

(4) home-based programs involving parents and children; 

(5) coordination with related programs; and 

(6) training of related personnel in appropriate skill areas. 
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Chapter Two 

Design of the Program and the Evaluation 



This section describes the basic design of Even Start programs as well as the design of 
the evaluation. Subsections include: 

• Even Start Program Design 

e Research Questions for the Evaluation 

• Components of the Evaluation 



Even Start Program Design 

The Even Start legislation mandates the major components of each local Even Start 
project. However, the legislation allows grantees great flexibility in devising projects to 
meet local needs. For example, Even Start encourages local staff to draw on available 
program models and to collaborate with existing service providers to create projects that 
are tailored to the needs of local families. Hence, Even Start can be regarded as a "family 
literacy laboratory" in which many different strategies are being tried. 

Exhibit 2.1 presents a conceptual model of Even Start, depicting the types of activities 
conducted and the causal chain anticipated as a result of those acn/ities. Projects are 
characterized as having a set of program inputs that influence program processes, which 
in turn produce outcomes for parents and children. At each level (inputs, processes and 
outcomes), a set of contextual variables act as mediators. Examples of measurable 
indicators are provided for each major set of variables shown in the model. 

The model shows that local projects can vary in many ways. For example, projects must 
make decisions about what services to provide directly and what services to provide 
through referrals, priorities for targeting children and adults (e.g., age of child, family 
language), the extent to which core services for families are to be integrated (e.g., 
whether activities in parent education should reinforce learning in adult education), 
whether to use an existing educational model and materials for delivering early childhood 
and adult basic education services, strategies for recruiting and retaining program 
participants, the role that parents play in the project, and staff development activities. 

Even Start projects are designed to use and build on existing educational and social 
services. Federal Even Start funds are to be used to coordinate existing services as well 
as to provide services which are not locally available. Many Even Start projects use case 
managers, parent liaisons, or family advocates as key staff in the provision of coordinated 
services. Case managers conduct needs assessments and have ongoing contact with a 
number of families at centers and through home visits. They are responsible for the direct 
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provision of some services as well as for ensuring that participating families take 
advantage of other services. 

Even Start law requires that projects provide three core services: 

• Early childhood education to meet the early education needs of children 
from birth through seven years of age, designed to enhance development 
and prepare children for success in school. 

• Adult education to develop the basic educational and literacy skills of the 
adult including adult basic education (ABE), adult secondary education 
(ASE), English as a second language (ESL), or preparation to attain a 
General Education Development (GED) certificate. 

• Parent education to enhance parent-child relationships and help parents 
understand and support their child's growth and development. 

ED regulations require that each family will participate in all three core services. Projects 
are free to choose the specific instructional strategies used in each of these core service 
areas. In addition, Even Start projects can decide to focus educational activities for 
children on a narrower age span than the birth through seven range that is required to be 
served by the legislation. Finally, Even Start projects are required to provide some core 
services to parents and children in joint sessions and to provide home-based services that 
are instructional in nature. 

Even Start projects must build on existing local services. Thus, depending on available 
sfirvir.Rs. anv or^all nf tha r.nrp. sarvir.fis mav rm nrnvidfirl hv staff fundfid thrnunta Fvp.n_ 
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It is also hypothesized that Even Start will have a positive impact on children's school 
readiness and school achievement. School readiness variables include age-appropriate 
cognitive, language, and social skills. Once children enter school, outcomes might include 
satisfactory school performance, and improved school attendance, as well as a lower 
incidence of special education, remedial placement, and retention in grade. 

What differentiates Even Start from Head Start and other "child-focused" programs is 
Even Start's dual focus on both adults and children in the same family. Even Start's 
hypothesis is that short- and long-term effects on children will occur both directly, 
through the provision of early childhood education services, and indirectly, through effects 
on their parents. 
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What is the relationship between amount of home-based services, amount 
of parent/child together services, length of participation, and outcomes for 
children? Outcomes for parents? 

How do parents' attitudes/expectations, basic skills, and patterns of parent- 
child interactions relate to children's school readiness or achievement? 

Do adults participating in Even Start have better retention and/or attendance 
in ESL or ABE programs than adults in regular adult education programs? 

Based on information about the services provided, is it possible to identify 
a set of Even Start "models" that exhibit variation in design and service 
delivery? Are some Even Start models more effective than others in terms 
of enhancing adult basic skills, children's school readiness, and parents' 
behaviors and expectations? Are some Even Start models particularly 
cost-effective? 

Across Even Start projects, are there practices or components that are 
particularly effective? 

In addition to the above questions, which were used to help design the evaluation, the 
Department of Education raised other research questions during the life of the project. 
Not all of these newer questions can be directly answered by data collected for this 
project. 

How well was the federal funding on the program spent? 
How many of the projects were well-implemented? 

What are project "best practices?" What types of projects or program 
elements work best under what conditions? 

How does the program compare to alternative programs addressing the 
same problem? Is it more effective? How do the costs compare? 

What is the program's impact on its target population and service delivery 
structure? 

How well does the basic Even Start model work? 



Components of the Evaluation 

A four-component evaluation was designed in order to address the questions listed above 
(see Exhibit 2.2). The components include: (1) the National Evaluation Information 
System (NEIS) for all Even Start projects, (2) an In-Depth Study of ten projects, (3) other 
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Exhibit 2.2 
Components of Even Start Evaluation 


Evaluation 


imputation w which 
data «•» collected 


typas of «kt» to b« 


Data coftactfoa 




Baafe research 
<WMtf>on» 


Analysis pkn 


Fimdioo 
■ BOUT**) for 

oofrMttian 


•OUrCa 

(or data 


National Evalu- 
ation Informa- 
tion System 
(NEIS) 


All Even Start 
projects and 
participants 


Participant charac- 
teristics, coordi- 
nation, services, 
implementation, 
costs 

Outcomes: school 
readiness, adult 
literacy skills, 
parent/child 
interaction 


Parent Quest., 
Family Service 
Log, Project 
Quest. 

Child and adult 
tests (PSI, 
PPVT, CASAS), 
parent 
interviews 


All years 
2nd-4th 


Who participated? 

How is the program 

implemented? 

How much service is 

received' 

What is the school 
readiness status of 
children? 

What is the literacy 
level of adults? Do 
adults attend adult 
education regularly? 


Descriptive 
analysis, review 
against Even Start 
goals 

Compare partici- 
pants' status and 
progress to norms 
and other programs 
for similar 
populations 


Local project 
evaluation 

Local project 
evaluation 


Abt/RMC 
Abt/RMC 


In-Depth Study 


Ever Start partici- 
pants and control 
group . 'om 10 
selectee Even Start 
projects 


Participant charac- 
teristics, coordi- 
nation, services, 
implementation, 
school performance, 
adult literacy, 
parent/child 
interaction, costs 


Observation;in- 
depth data on 
participants and 
servicesiparent 
and staff inter- 
views; design 
for longitudinal 
study of chil- 


2nd-4th 


What are the short 
and long-term effects 
of Even Start on 
children, parents, and 
families? What 
models work best? 
What aspects are key 
to success? 


Compare against 
control group data 
and against data 
from other national 
programs 


Abt and local 

project 

evaluation 


Abt 


Local models 
evaluation 
(PEP/NDN 
qualification) 


Even Start projects 
that qualify 

Even Start 
participants and 
comparison/control 
group, if appropriate 


Participant charac- 
teristics, coordina- 
tion, services, imple- 
mentation, costs 
school readiness, 
adult literacy, 
parent/child 
interaction 


Testing and 

parent 

interviews 


3rd-4th 


Is the project 
exemplary? Is it a 
transferrable model? 


Compare gains 
within the project 
to those of similar 
local families or to 
national norms 


Local project 
evaluation 


Local 
project 
evalu- 
ation 


Othpr locul 
evaluation 
naodu 


Conducted at local 
lovel 


Additional 
information desired 
by local 
administrators 


Proposod in 

project 

application 


All yoars 


How does the project 
meet specified local 
noeds? 


Depondt on 
questions 

I 


Local project 

evaluation 

budget 


Local 

projoct 

ovalu 

ation 

budget 
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Exhibit 2.3 

Summary of Cohort 3 and 4 Data Submissions 


Form of Submission 


Percent of Projects 


On-Disk Only 


53% 


Hard Copy Only 


14% 


Both Disk and Hard Copy 


16% 


Serving No Families 1 


6% 


Late/No Submission 


1 1 % 


TOTAL 


100% 



'These were projects in their first year that received funding late in the year and were not yet enrolling 
families by. the date of data submission. 



The In-Depth Study was implemented with 10 projects, purposively 
selected from the first cohort of 73 Even Start grantees. 

Projects were selected based on a number of criteria, including 
geographic location, level of program implementation (e.g., providing 
full range of Even Start services, program fully operational) and 
willingness to participate in the In-Depth Study. 

Families were randomly assigned to Even Start or to a control group 
in as many of the In-Depth Study projects as possible. Five of the 
ten In-Depth Study projects implemented random assignment. 
Where random assignment was not possible, no comparison group 
was assigned. The five non-randomized projects contributed 
information to the descriptive and cost components of the study. 

The In-Depth Study plan called for about 20 Even Start families and 
20 control group families in each project. In total, sample sizes were 
about 200 Even Start families (20 per project for ten projects) and 
100 control group families (20 per project for five projects). 

The In-Depth Study sample focused on families with children three 
or four years old. Project staff used this as a criterion for recruiting 
families to participate in the study. 

Recruitment of families occurred in the summer/fall of 1991, 
assignment to grouos and pretest data collection occurred in the fall 
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of 1991 through January 1992. Posttest data were collected in the 
summer of 1992 and again in the spring of 1993. 

• Site visits by Abt staff were conducted in the spring of 1991 (to 
describe programmatic activities), in the spring of 1992 (to describe 
program costs), and again in the spring of 1993 (to obtain project 
director's reflections on Even Start). 



Other Local Evaluation Activities 

After grantees met requirements for the National Evaluation Information System and the 
In-Depth Study, they were free to conduct other local evaluation activities. Local 
evaluation activities could be funded through the projects' evaluation budget, but had to 
be approved by the Department of Education, typically through the continuation grant. 



Local Application for PEP/NDN Qualification 

The final component of the evaluation was primarily the responsibility of individut Even 
Start grantees. In accordance with Section 1058(c) of the Even Start legislation, goal 
of Even Start was that projects should submit evidence of their effectiveness for approval 
by the Department of Education's Program Effectiveness Panel (PEP). Approval by PEP 
results in entry to the National Diffusion Network (NDN) and national recognition as a 
model program worthy of emulation. After entry to NDN, the project may apply to NDN 
for additional dissemination funds as a developer/demonstrator project. 

Starting in 1 993, Even Start projects also could apply to a new NDN dissemination center 
on family literacy in order to obtain approval and recognition of their program. The 
National Center for Family Literacy in Louisville, Kentucky now has authority to identify 
exemplary family literacy projects for inclusion in the National Diffusion Network. 
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Chapter Three 

Description of Even Start In -Depth Study 
Projects 



To help readers understand how Even Start projects operate, this sect : on describes the 
characteristics of participating families in each of the ten In-Depth Study projects as well 
as the programmatic activities undertaken in each project during the 1990-91 program 
year. More complete descriptions appear in the technical appendix to this report. 1 This 
section includes the following topics: 

• Characteristics of Families in In-Depth Study Projects 

• Overview of Project Activities 

• Synthesis of Implementation Activities and Issues 

• Changes in Service Delivery Strategies from 1991 to 1993 



Characteristics of Families in In-Depth Study Projects 

Exhibit 3.1 summarizes the characteristics and number of families participating in the ten 
In-Depth Study Even Start projects during the 1990-91 program year. These projects 
serve families with a range of racial/ethnic characteristics, including Hispanic, black, 
white, Native American, Hmong, and Afghani families. Four projects (Birmingham, 
Billings, Reading, and Richmond) have the capacity to accommodate between 30 and 50 
families in the program at one time. Three projects (Phoenix, Indianapolis, and Estill) 
serve 60 to 70 families. Golden and Albuquerque are larger projects, with 90 to 100 
families participating, and Waterville is the smallest program with 20 to 25 families 
maximum. 

Exhibit 3.1 also shows the ages of the children targeted for educational activities. Most 
projects offer some activities or services for all children in the full Even Start age range 
as well as special events for all family members. Some of the projects focus on a 
narrower age range for structured educational activities with children. Typically, projects 
offer classes for preschool children who are between three and five years of age. 



1 These descriptions were written based on 3-day site visits to each of the 10 projects 
during the spring of 1991. 
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Education Center. Home visits take place once a week for an hour; parent meetings or 
"group home visits" take place once a month at one of the housing projects. 

Phoenix, Arizona. Program activities revolve around preschool classes for children ages 
three and four that are held in portable classrooms on the campus of the Butler 
Elementary School. Parents volunteer in their child's classes at least twice a month; 
attend parent workshops at least twice a month; and participate in "Read-to-Me" sessions 
once a month. ESL and GED classes are provided in the evenings by the Rio Salado 
Community College, with Even Start staff providing child care. For adults who are 
reluctant to go to classes or not ready for GED instruction, there is one-to-one literacy 
tutoring available through the local chapter of the Literacy Volunteers of America. Home 
visits are conducted by the early childhood education teachers and aides once a month 
for families of four-year-olds and once a week for families of three-year-olds. 

Golden, Colorado. This Even Start project is based on a case management model in 
which five parent liaisons work with 18 to 20 families each to help adults participate in 
adult basic education classes and enroll children in early childhood education classes. In 
addition, the liaisons plan and implement parenting activities during biweekly home visits. 
The project collaborates with the Jefferson County school district to provide adult 
education, paying tuition for adults to attend the district's ESL, GED preparation, or high 
school diploma programs. Preschool age children attend one of the district's Language 
Development preschoois, Head Start, or day care. 

Indianapolis, Indiana. The Indianapolis Even Start project is an adaption of the Kenan 
Trust Family Literacy Project model where parents and children attend educational 
programs at the same site. Parents and children attend classes four afternoons or four 
mornings a week. While parents are in adult basic education classes, their three- and 
four-year-olds are in a preschool based on the High/Scope curriculum. Parents spend one 
half-hour in their children's classroom each day working on activities together. One hour 
per week parents meet with a social worker for a group discussion on parenting issues 
or to hear a guest speaker. Home visits are conducted on Fridays by two-person teams; 
each family receives a home visit about every six to eight weeks. 

Waterville, Maine. This is primarily a home-based project. Adult basic education, 
parenting, and parent-child activities are all provided by home visitors, with the curriculum 
tailored to the needs of each family. All together, each family is visited for between four 
and ten hours a week by two or three different visitors. There also are monthly potluck 
suppers for the whole family and parent discussion groups without the children. The 
project collaborates with Head Start to provide a structured early childhood program. 

Billings, Montana. The project offers educational activities for children from birth through 
age five at the Even Start center. While children are in classes, their parents receive adult 
education at the school district's Adult Education Center a few miles away. The program 
also works closely with the JOBS program, and offers child care and parenting classes 
for JOBS participants who are taking adult basic education classes at the adult high 
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Albuquerque, New Mexico. Project activities center around the Even Start sites at two 
elementary schools. At each site, there are half-day preschool classes twice a week for 
children ages three to five. For ch.dren one and two years old, the early childhood 
education teachers do home visits. Adult education options include ESLand GED classes 
at the Even Start sites provided by staff from the Albuquerque Technical-Vocational 
Institute and Southwestern Indian Polytechnical Institute. In addition, tutoring is offered 
by the Albuquerque Literacy Program, an affiliate of Literacy Volunteers of America. 
Parents have a choice of activities to consider in order to complete the parent education 
component, including monthly parent meetings and volunteering in their child's classroom. 

Reading, Pennsylvania. The Reading Even Start Project offers activities for parents and 
children at four community sites, including elementary schools and a local community 
college. Parents attend GED and ESL classes while their children are in the Even Start 
early childhood education component. The project serves children in the full range from 
birth through age seven, and all of the classrooms have mixed age groupings. Classes are 
offered in the morning or afternoon three or four days a week, depending on the site. 
Parent education takes place during parent discussion groups, parent-child time in the 
classroom, and home visits. 

Estill, South Carolina. The Estill Even Start project is structured sequentially in four time 
cycles. Each cycle has a different emphasis: Cycle One, offered during October and 
November, focuses on parenting; Cycle Two runs from November through January and 
provides Life Skills; Cycle Three provides computer skills and literacy/GED training from 
January to June; and Cycle Four is the summer program offered during June and July. 
The project focuses on children who are four and five years old. Children attend either 
the district kindergarten or Head Start classes while their parents participate in the various 
nine-week segments. Even Start staff conduct monthly home visits to check in with 
families and share instructional materials. 
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Adult Education 



Adult Education Activities. Even Start projects offer a range of activities designed to 
enhance adult basic skills. Almost all projects report that they provide services for adults 
to attain a GED certificate; a majority of projects also offers services in adult basic 
education and adult secondary education. Many projects offer services in English as a 
second language. 

The ten In-Depth Study projects offer a range of activities to enhance adult basic skills. 
All programs have special features that they use to augment their adult education program 
and make it more responsive and appealing to adult learners. Exhibit 3.2 lists some of the 
special features of the adult education programs in these ten sites, including computer 
labs, literacy volunteers, writing programs, academic counselors, parent newsletters, 
home-based instruction, and credit for life experiences. 

A number of projects incorporate computers into their instructional program for adults. 
In Birmingham and Estill, adults use the IBM PALS program. The PALS curriculum is a 
1 00-hour instructional program designed to teach computer and literacy skills to students 
who read at or below the fifth-grade level, in Indianapolis, the adult education classes 
use computers purchased with a grant from the National Center for Family Literacy and 
Apple Computers. Students use the computers for math, geography and word-processing 
skills. The computers are linked to the Even Start project in Oregon, which also has an 
Apple grant, and the two projects have instituted a pen pal program among adults. In 
Reading, adult learners use computers to create monthly newsletters. 

Several of the Even Start programs incorporate real-world reading materials into adult 
education classes. In Indianapolis, the adult classroom receives USA Today that students 
read daily and Classline Today , a one-page list of questions and projects linked to USA 
Today that students complete as homework. The Even Start project in Albuquerque 
receives from local publishers 100 copies of newspapers that they send home with the 
children. The project has run workshops for parents on how to use the paper and tries 
to get two pieces of literacy materials into the home each week. The "Life Lab" in the 
Richmond project illustrates another way that newspapers can be used to encourage 
reading and thinking skills. 

Adult Education Challenges. One of the challenges of adult education expressed by staff 
is the extent to which it focuses on GED preparation. Some programs offer adult 
education classes specifically geared to the GED test and incorporate GED workbooks into 
adult education classes, while others provide general instruction in basic skills such as 
reading, writing and math. Some project staff have expressed concern that attainment 
of the GED certificate is seen as a key outcome of the success of Even Start, yet for 
many adults the GED is a long-term or possibly even unrealistic goal. 

When to offer the adult education activities also has been a challenge for some projects. 
A number of sites offer adult education classes during the day when children are in early 
childhood classes. This option has several advantages: transportation arrangements can 
be coordinated for adults and children; the early childhood program may meet parents' 
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Writing a Newsletter in Reading, Pennsylvania 

The adult education classroom at the Amanda Stout elementary school 
is located next to the children's classroom and has several rectangular 
tables where parents work. Five women are working on a monthly 
newsletter. Two women are sitting together at one table writing their 
stories out on lined paper and referring to the dictionary placed on the 
table between them for help with spelling. Another woman is working 
on an Apple computer, entering her story and selecting the graphics to 
accompany the words. The women are chatting in Spanish but writing 
in English. The teacher moves from one table to the next, offering help 
and suggestions to the adult learners. 

On the wall of the classroom are sample stories from previous 
newsletters. In some articles, parents write about outings with their 
children and weekend visits to family members, illustrated with pictures 
of butterflies and kites. Other articles include personal descriptions and 
family recipes. All of the articles have bylines or other statements 
identifying the authors. 



Parent Education 

Parenting Education Activities. Parent education activities include group discussions, 
hands-on activities, home visits, and guest speakers. Projects tend to use materials from 
a variety of curriculum sources rather than relying on one source exclusively. Some of 
the commercially available materials that are used include Bowdoin Parenting Education, 
Systematic Training for Effective Parenting (STEP), Dorothy Rich's "Megaskills," and Head 
Start's "Looking at Life." The frequency of activities varies across the ten In-Depth Study 
sites from one hour a week of regularly scheduled activities to special parent meetings 
once a month. 

Parent-child activities and home visits also enhance parents' knowledge and skills. Exhibit 
3.3 summarizes the frequency and types of parent-child activities offered in the ten In- 
Depth Study sites. Examples include center-based activities with parent volunteers in 
classrooms, pot luck suppers, and evening meals. 
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Exhibit 3.3 



Parent-Child Activities in In-Depth Study Projects 
(1990-91 Program Year) 



Project 


Center-Based Activities 


Home-Based Activities 


Birmingham, AL 


Volunteer in the ECE 


Weekly home visits; monthly 




classroom once/month 


"group" home visits at 






housing project 


Phoenix, AZ 


Monthly "Read-to-Me" 


Weekly home visits for 




sessions; volunteer in ECE 


parents of 3-year-olds; 




classroom twice/month 


monthly home visits for 






parents of 4-year-olds 


Golden, CO 




Home visits twice/month 
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"Life Lab" in Richmond, Virginia 

Parents start the first half-hour of each day in "Life Lab," where they 
read the morning paper. The class is divided into three groups, each 
with four or five women sitting around a table. Every adult has a copy 
of the Richmond daily newspaper and each table has a dictionary. 
Participants help themselves to coffee from a large pot on one side of 
the room. 

The groups discuss different issues presented in newspaper articles. At 
two of the tables, adult education teachers serve as facilitators. At the 
third table, the Even Start adults are running the discussion themselves 
because one teacher just left on maternity leave. As one group reads an 
article about taxes, the adult education teacher offers information about 
different types of taxes and the uses of this money for public services. 
Another tables is reading an article about a local crime. Morning 
discussions may range from current events to comments about sales or 
coupons at local stores. Each student takes the paper home at the end 
of the day. 



Phoenix runs monthly "Read-to-Me" sessions, described below, that serve multiple 
purposes of emphasizing reading, encouraging parents and children to spend time together 
in educational activities, and getting more reading materials into the families' homes. 

Home visits provide an opportunity for project staff to interact with parents and children 
in a less formal way than in a structured classroom. Many staff described the benefits 
of going to families' homes, such as getting to know the whole family and letting parents 
know that they care enough to come to their home. All of the In-Depth Study projects 
conduct home visits, with the exception of Richmond. These home visits generally have 
multiple purposes that include modeling for the parent some educational activity with the 
child, leaving toys and books in the home, and maintaining contact with families. A 
description of a home visit in Indianapolis is presented below. 

A number of projects encourage or require parents to volunteer in their child's classroom 
which fosters positive interaction with their own children and use of appropriate activities 
to enhance their young children's development. Three of the projects incorporate Parent 
and Child Together (PACT) time, a feature of the Kenan model of family literacy programs, 
into their program and have parents spend between 30 and 45 minutes every day in their 
child's classroom working on activities together. In Indianapolis, the joint parent-child 
activities for PACT time are chosen either by the child, the parent, or the teacher. 
Because staff realized that parents became bored when their children always select the 
same activity, they have parents select the activity on one day, the children on two days, 
and the staff on one day each week. In Richmond, the specific activities that parents 
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work on during PACT time are planned by the early childhood teachers in concert with the 
adult education teachers. The children also have some say in the specific activities and 
often choose activities before their parents arrive. 



A "Read-to-Me" Session in Phoenix, Arizona 

The parent-child activity, led by two early childhood education teachers, 
is attended by about 1 5 mothers and 20 children. The mothers and their 
children sit together around tables in the preschool classroom. Younger 
siblings are in the next room with three classroom aides. The theme is 
"peek-a-boo" books (i.e., books where someone or something is hiding 
either literally under a flap in the book or somewhere in the picture). 
The teachers read three books to the group, going page by page and 
asking questions along the way, encouraging children to get involved 
(and they do-calling out answers, telling where objects are hiding). The 
session is a mixture of English and Spanish. 

After the stories are read, the parents and children make their own peek- 
a-boo books to take home. Parents have the choice of making a book 
with English or Spanish text. The teachers explain to the parents how 
to make the book, pass out all the materials that they need, and 
encourage the parents to let it be a joint activity with their children. The 
materials include paper printed with a sentence identifying a hidden 
object, "flip-up" pieces of paper that the teachers and project director 
made with a die-cut machine, and stickers of the objects named on the 
page. The goal of the session is to have the parents read the text to the 
child and have the child select the sticker that matches the text. 

When all of the pages are complete, the mothers use yarn to bind the 
pages into a book. This turns out to be a difficult task for some 
mothers, and there is a lot of discussion and help rendered by mothers 
at the same table. The teachers also circle around to help. The session 
lasts about an hour, with parents and children taking their new book 
home. 



Parenting Education Challenges. Staff from several projects discussed the profound need 
that participating families have for social services. Some projects employ social workers 
to address these needs. In other sites, staff felt that it would be a great help to have a 
professional social worker available for families. In many sites, staff talked about the extra 
roles and responsibilities they take on to help families negotiate with local educational, 
medical, or social service agencies. Staff also discussed their concerns about the need 
to help families without "coddling" them, and the importance of being open with families 
while keeping some professional distance. 
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It seemed clear that staff were compassionate and wanted to help families overcome the 
many problems they face, yet additional roles beyond teacher or home visitor could easily 
become overwhelming for staff. This is an area where Even Start project directors must 
make important resource decisions. Some projects have chosen to hire social workers 
specifically to deal with social service needs, while other projects expected that these 
tasks could be assumed by staff with training in early childhood education or adult 
education. It is not clear which approach is more effective, and it is likely that the best 
approach for a given site depends to a great extent on the characteristics of the 
population being served. 

Projects that include joint parent-child time in the early childhood classroom described this 
activity as both a benefit and a challenge of the program. As one early childhood teacher 
commented, parent-child time is the "heart" of the program that makes it unique and, 
without it, "this would be just another program for disadvantaged children." Yet at the 
start of the year,, staff report it is hard to get parents and children to play together 
because parents <nay be uncomfortable or unsure of how to play with their children or 
reluctant to let children do things like using scissors on their own. 



Early Childhood Education 

Early Childhood Education Activities. The Even Start projects in Phoenix, Indianapolis, 
and Richmond base their preschool classrooms on the High/Scope curriculum which 
centers around key areas of cognitive development such as language, representation, 
classification, seriation, numbers, spatial relations, and time. The district preschools with 
which Estill collaborates also are based on High/Scope. The curriculum gives children the 
opportunity to make choices at all times by building classroom activities around a 
sequence of "Plan/Do/Review": "Plan" is when children choose their activities, "Do" is the 
work part of the day, and "Review" is recalling what activities took place. The 
High/Scope curriculum also encourages the use of "life size" materials in the classroom 
such as real cereal boxes, telephone books and full-size pots and pans in the 
housekeeping area. 

Another manifestation of the High/Scope model is the use of labels and symbols in the 
room. For example, in Indianapolis, on the shelves that hold art materials, there are 
pictures of crayons under the crayon boxes; on the shelves where the blocks are stored, 
there are different shapes of paper taped to the shelves to correspond to the various 
shape blocks. The symbols help children put things back in the right place and also 
reinforce cognitive matching skills. 

The early childhood classes in Birmingham and Albuquerque incorporate similar planning 
strategies. In Bin lingham, the preschool children choose play centers such as the sand 
table, kitchen area, or manipulative toys. The teacher specifies how many children can 
be in each center and the children tell her where they want to work. The teacher puts 
on music during play time and the children know that when the music is turned off, it is 
time to clean up from that activity and gather for group story reading. In Albuquerque, 
classroom activities revolve around small group activities, called "committees," that the 
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Exhibit 3.4 




Staffing Arrangements Among In-Depth Study Sites 
(1990-91 Program Year) 




Project 


Adult Education 


Parent Education and 
farent-uuld Activities 


Early Childhood 
Education 


Birmingham, AL 


District teachers 


Even Start facilitator; 
district parent education 
coordinator; Even Start 
home visitors 


Even Start teachers 


Phoenix, AZ 


Community college 
teachers; literacy 
volunteers 


Even Start ECE teachers 


Even Start teachers 
and aides 


Golden, CO 


District ABE teachers 


District ABE teachers; 
Even Start parent liaisons 


District preschool 
and Head Start 
teachers 


Indianapolis, IN 


Even Start teachers 


Even Start social 
workers; ABE teachers 
and ECE teachers 


Even Start teachers 
and aides 


Waterville, ME 


Even Start home 
visitors; district ABE 
teachers 


Even Start home visitors 


Head Start teachers 


Billings, MT 


District teachers 


Even Start family 
advocates 


Even Start teachers 


Albuquerque, NM 


Community college 
teachers; literacy 
volunteers 


Even Start ECE teachers 


Even Start teachers 
and aides 


Reading, PA 


Even Start teachers; 
community college 
teachers 


Even Start ABE and 
ECE teachers and home 
visitor 


Even Start teachers 
and aides 


Estill, SC 


Even Start teachers; 
district ABE teachers 


District teachers; Even 
Start project director and 
social worker 


District and Head 
Start teachers 


Richmond, VA 


Even Start teachers 


Even Start ABE teachers; 
Richmond Early 
Learning Center teachers 


Richmond Early 
Learning Center 
teachers and aides 
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the floor, turning off the TV, and keeping interruptions to a minimum. The program also 
makes a special home visit to families that have missed four classes without a reason; if 
attendance does not improve, a letter is sent telling the family they will be dropped from 
the program if they do not come to classes. While project staff are always reluctant to 
have families drop out of the program, they recognize it is necessary when families are 
no longer actively participating in order to offer that slot to other interested families. 

Projects also provide tangible rewards for participation. In Birmingham, where staff found 
that attendance was hardest to maintain during the summer, parents were given prizes, 
such as a fan or books, for participating in parenting classes. In Albuquerque, in addition 
to giving out newspapers, evening parent meetings included a brief award ceremony 
where parents are given t-shirts for perfect attendance. In Richmond, parents who 
maintain good attendance for two to three weeks get a paperback dictionary to take 
home. On-time attendance for two days in a row also earns a $1 certificate in the Even 
Start "store" where parents can buy deodorant, detergent, toothpaste, vitamins, and 
other items they have requested. 

Other incentives to participate are woven into program activities. For example, in Estill, 
where families come from a distance of 30 miles to the classes, the evening sessions 
start with a dinner where parents, children, and staff eat together. In Indianapolio, the 
project stresses the importance of recognizing that parents are students. The project has 
set up a separate lounge where they can smoke and has installed vending machines 
outside the adult basic education classroom so that parents can bring soda and snacks 
into class. Reading established a Parents' Council, which consists of two parent 
representatives from each of the four sites, that meets every two months with the 
program administrative aide/van driver and the home visitor to give parents a chance to 
express their concerns and offer suggestions. 

Special events are another way that programs try to connect with families. In Phoenix, 
a Saturday family outing, paid for by the Coors Literacy Foundation, attracted about 220 
people. A local puppet theater provided entertainment, local newspaper women showed 
the adults how to use the newspaper in everyday life and each family was given a year's 
subscription to the local paper (worth about $100). In addition, a church gave each 
family three books. 

Two projects have started a mentoring program linking parents with adults from the 
community. Indianapolis began a mentoring program as a way to provide as much 
support as possible to adults in the program, to provide positive role models, and to 
increase retention. The project developed a packet of information that describes the 
mentoring program, outlines the role of the mentor for parents, the parent's 
responsibilities, the guidelines and ground rules for mentors, and the rationale for the 
mentoring program. For example, the materials tell both the parents and the mentors that 
the relationship is one of sharing thoughts and feelings and should not involve loaning 
money or buying things for the families, and informs the mentors that any concerns about 
the families or need for social services should be directed to the Even Start social 
workers. In Richmond, a few students meet with black women who are part of a 
mentoring program from the Junior League and talk about job skills. 
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Changes in Sendee Delivery Strategies from 1991 to 1993 



A set of one-day follow-up site visits was conducted in the spring of 1993, two years 
after our initial visits to the In-Depth Study projects. We found that during this period, 
all projects had refined, modified, and revamped program activities in order to improve the 
services offered to families. We summarize below the areas in which projects changed 
their service delivery-ranging from the content of adult or early childhood education to 
the processes by which families participate to the staff who provide services. 



Content of Core Services 

In all In-Depth Study sites, project staff agreed that the mission of their program has 
remained the same, but that they continue to find ways to improve services provided to 
families. For example, in Reading, the early childhood education staff began to use the 
Child Observation Record system developed by High/Scope, and several of the early 
childhood staff have rearranged their classroom environments after attending a 
High/Scope workshop. In Billings, the staff-child ratio has improved, which has allowed 
the staff to introduce more variety into the educational activities, especially with the 
younger children. The Estill project has added a program component for older siblings 
that focuses on building self-esteem; the group meets weekly with the social worker. 

In many projects, staff continue to work to align parenting education more closely with 
other core services by scheduling debriefing meetings for the project staff to discuss 
families' needs, or to integrate the content of early childhood education lessons into 
parenting discussions more completely. The Birmingham project conducts an annual 
needs assessment to learn from the parents what they would like to have covered in 
parenting education sessions. 

h Richmond, the adult education staff has built more flexibility into the schedule for 
parents so they can choose which subjects to study. Albuquerque, Phoenix, and 
Birmingham are continuing to work with collaborators who provide adult education 
services to improve the fit between what other agencies provide and what the Even Start 
projects would like for their adult participants. 



Recruitment and Retention 

For most sites, recruitment has become much less of a chronic concern because projects 
have established networks within their local communities and are known to families and 
schools as well as to other service providers. In several sites (Birmingham, Golden, 
Billings, Richmond), close collaboration with local JOBS programs has increased the pool 
of potential participants, and has eased recruitment challenges. But for two projects, 
recruitment has been an issue over the past year, because the projects themselves are 
uncertain about their future. Project staff in Golden struggle with decisions about 
recruiting participants who could be meaningfully served in a year's time, and Waterville 
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staff are focusing their efforts on the families already participating in the project rather 
than adding new families. 

Retention was mentioned by several projects' staff as an ongoing issue. A number of 
staff commented that the more mature the parent the greater the likelihood of a 
meaningful commitment to participate. One strategy used to increase retention is to 
establish attendance requirements, and families who do not meet the standard are 
dropped by projects. Staff also mentioned that because recruitment has become less of 
an issue, it has been easier to drop those families who have not met their commitments. 
Some projects also provide incentives to children and parents for good attendance. 
Parents at the Reading site developed a system whereby parents must notify other 
parents--who serve as monitors-if they are not able to attend a session; this has helped 
improve attendance a great deal. 



Stqffing 

The project directors across all ten sites have remained in their jobs, although there has 
been one change in the project director position, in Billings. Other staff have also 
remained fairly stable, with some attrition but with a core group of staff remaining. In all 
10 sites, there has been a group of core project staff for the duration of the project, 
which contributes to the projects' continuity. 



Strengths and Challenges 

Staff discussed several strengths of the Even Start program, including: 

• The flexibility of the Even Start model -- because the model itself can 
change in response to participants' needs, the model can continue to 
serve its participants. 

• The family-centered focus of the program which can accommodate 
needs of parents and children simultaneously. 

• The collaborations -- projects are able to pull resources together for 
families in ways that other individual service agencies had not been 
able to do. 

• Project staff are viewed as a source of strength for the project. 

• Home visits -- staff in projects that rely upon the case management 
approach (Billings and Golden) or use home visits as a primary mode 
of service delivery (Waterville) believe that the home visit component 
represents a significant advantage of the Even Start model. 
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• The length of the program allows participants and staff the 
opportunity to establish meaningful relationships; families who have 
not easily trusted service providers have a chance to become 
comfortable with educational institutions. 

Challenges implementing the Even Start program include: 

• Support services -- projects cannot afford to provide complete 
coverage in such areas as child care or transportation and it is 
difficult to get these services through collaborations. This is more of 
a problem in projects that operate in multiple sites or on a staggered 
schedule. 

• Space is difficult to find, particularly in oversubscribed schools and 
community centers, not only in terms of sufficiency but in terms of 
appropria + eness for either young children or for adults. 

• Heterogeneity of participants -- the ages of children range from 
infancy to preschool, and teachers have to cope with widely 
divergent developmental levels. For adults, teachers have to work 
with students with varied backgrounds in English language 
proficiency, in goals for participation in Even Start, and in 
commitment to the project. 

• Uncertainty of the future -- several directors commented that they 
have encouraged their staff members to seek other employment 
because they could not offer any job security. 



Effectiveness of the Project 

A reduced need to recruit actively was mentioned by nearly all project directors as one 
indication of effectiveness. This translates into having to spend less effort recruiting 
participants as well as less effort recruiting potential collaborating agencies. Many project 
directors noted that the program is more effective for parents who are more mature, and 
who are ready to make a commitment to change. 

In Estill, a project that targets 4- and 5-year-olds, kindergarten teachers have remarked 
that Even Start children are doing noticeably well in school, and that fewer Even Start 
children are being retained in grade or are failing the compulsory exam for first-graders 
than might be expected. Other projects also reported positive comments from 
kindergarten teachers about how well-prepared the children are and that parents have 
become involved in their children's education. 

A number of project directors also commented specifically about changes in families-that 
fewer families had moved away (especially in areas with traditionally high mobility), that 
parents were taking responsibility for their own behaviors and for their children's 
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behaviors, and that children were presenting fewer problems in school settings. One 
collaborator commented that because the Even Start program is still changing, and 
because it has not yet become institutionalized like Head Start, it can respond to local 
needs much more effectively than can Head Start or other more established programs. 



Relationships with Local School Districts 

Every project director described the local school district as supportive of Even Start. 
Some staff described relationships with the local Chapter 1 office and staff as more well 
developed than with the classroom teaching staff because of the connections between 
Even Start and Chapter 1 within the central office, and also because not many children 
have yet entered first grade. In Reading, the project director reported that the project has 
a strong relationship with the state Chapter 1 office in addition to its good relationship 
with the local Chapter 1 office. In Phoenix, the preschool teachers and kindergarten 
teachers have an annual "Switch Day" so each group can appreciate what the other 
routinely faces. 

One widespread comment was that Even Start has begun to influence what the schools 
are doing. The project director in Golden reported that Even Start has helped schools 
redefine what their mission is, and she takes that as a reflection of Even Start's success 
in working with hard-to-reach . families. Administrators in Reading, Richmond, 
Indianapolis, Phoenix, and Albuquerque concurred that the Even Start model of working 
with the entire family has helped convince school districts of the importance of 
establishing relationships with families before their children begin kindergarten. In 
Waterville, classroom teachers have begun to make home visits; in Golden, the Chapter 



projects were concerned about the longer-term funding for their projects and the local 
district's or the state's commitment to inter-generational literacy programs. What came 
across from the entire group was that staff and collaborators alike believe that the Even 
Start program has a future because the need for services remains. 
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Chapter Four 



Characteristics of Even Start Participants: 
Who is Served by Even Start? 



This section presents data from the NEIS that describe the families, children, and adults 
that participated in Even Start during the 1992-93 program year in terms of household 
composition, race and ethnicity, educational and employment status, primary language, 
and other variables. There were 340 Even Start projects operational in 1993-93. Some 
exhibits are based on 1 20 cohort 1 and 2 projects; others are based on 270 out of 340 
projects across the four cohorts. The effects of these missing data are unknown. 
Subsections include: 

Number of Participants 
Characteristics of Participating Families 
Characteristics of Participating Adults 
Characteristics of Participating Children 
Comparison with Head Start and CCDP 
Characteristics of LEP Adults 
Need Level of Even Start Adults 
Project Variation in Population Served 



Number of Participants 

Even Start provides three types of educational services (adult education, parenting 
education and early childhood education) and several types of support services (e.g., 
transportation, meals and counseling). During 1992-93 core services were provided to 
16,518 families, an average of 61 fam 'ies per project. Projects provided early childhood 
education services to 22,429 children (83 per project), and adult education or parenting 
education services to 18,586 parents (69 per project) in these same families (Exhibit 4.1 ). 



Characteristics of Participating Families 

Family characteristics discussed here a e based on data from families in which at least 
one family member participated in some Even Start core service during the 1992-93 
program year. 
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Exhibit 4.3: Number of Adults in Even Start Families 
(1992-93 Program Year) 



Three a 
Four or 




Two adults 



Exhibit reads: 33.2 percent of Even Start families had one adurt in the household. 
Note: Based on reports from 270 of 340 projects. 



Exhibit 4.4: Number of Children in Even Start Families 
(1 992-93 Program Year) 



One child 

/ 

j 




Two children 



Three children 



Exhibit reads: 20.9 percent of Even Start families had one child in the household. 
Note: Based on reports from 270 of 340 projects. 
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Exhibit 4.6 

Primary Sources of Financial Support for Even Start Families 
(1992-93 Program Year) 



Primary Sources of Financial Support Percent of Families 



Government assistance 49% 

Job wages 46% 

Alimony/child support 2% 

Other 3% 



Exhibit reads: 49 percent of Even Start families relied on government assistance as their primary source 

of financial support in 1992-93. 

Note: Based on reports from 270 of 340 projects. 



Exhibit 4.7: Total Annual Income of 
Even Start Families (1 992-93 Program Year) 



$5,000-$9,999- 




$25,000 or more 

— $20,00O-$24,999 

i,000-$1 9,999 



Exhibit reads: 35.1 percent of Even Start families had annual incomes of less than $5,000 in 1992-93. 
Note: Based on reports from 270 of 340 projects. 
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Characteristics of Participating Adults 



This section describes the more than 18,000 adults who participated in Even Start core 
services during 1992-93. 



Age of Participating Adults in Even Start Families 

Most adults in Even Start families were between 22 and 29 years old (46 percent), or 
between 30 and 39 years old (31 percent). Only 13 percent were in the 18 to 21 age 
range, 3 percent were younger than 18, 6 percent of Even Start adults were 40 to 49 
years old, and 2 percent were 50 or older (Exhibit 4.8). This age distribution has not 
changed over time. Given the population targeted by Even Start, it might be expected 
that more than three percent of Even Start adults would be under 18 years of age. 
However, a family is eligible for Even Start only if an adult in the family qualifies for 
adult basic education, and adult basic education participants must either be at least 16 
years old and not in school or beyond the age of compulsory schooling in their state. 



Exhibit 4.8: Age of Participating Adults in Even Start Families 
(1 992-93 Program Year) 




Exhibit reads: 13.4 percent of participating adults in Even Start families were 18-21 years old in 1992-93 
Note: Based on reports from 270 of 340 projects. 
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Exhibit 4.10: Racial/Ethnic Background of Participating Adults 
in Even Start Families (1 992-93 Program Year) 



Asian, Pacific Islander, Other 
Native American 



Hispanic- 




African- American 



White 



Exhibit reads: 25.6 percent of participating adults in Even Start families identified their racial/ethnic background 
as African-American. 

Note: Based on reports from 270 of 340 projects. 



Exhibit 4.1 1 : Ethnic Background of Hispanic Adults in Even Start Families 

(1992-93 Program Year) 



Other Hispanlc- 



Cuban- 
Puerto Rlcan 



16.4% 




-Mexican 



Exhibit reads: 74.8 percent of Hispanic adults in Even Start identified their background as Mexican. 
Note: Based on reports from 270 of 340 projects. 
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Exhibit 4.12: Racial/Ethnic Background of Participating Adults 
in Even Start Families, by Program Year 

50% -j 

|45%! l45%j 
. — , , — ,j mitB 




I 



0% h , 1 , 

1989-90 1990-91 1991-92 1992-93 



Exhibit reads: The percentage of African-American adults participating in Even Start dropped from 36 percent 
in 1990-91, and then remained constant. 

Note: Based on reports from 1 20 projects in 1 989-90, 1 990-91 , and 1 991-92, and from 270 of 340 projects 
in 1992-93. 



adults to 45 percent and decreasing the percentage of African-Americans to 26 percent. 
These numbers were unchanged in 1991-92. However, in 1992-93, the percentage of 
white adults returned almost to its initial level (40 percent), while the percentage of 
Hispanics and Asians/Native Americans/others rose to 22 percent and 1 3 percent. There 
were no federal policies that would lead to such changes, and we assume that the 
observed changes are due simply to differences between the racial composition of newer 
projects and older projects. 



Educational Attainment of Adults in Even Start Families 

A distribution of years of education prior to participating in Even Start is shown in Exhibit 
4.13. The solid line represents adults participating in any type of Even Start core service. 
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Exhibit 4.13: Years of Schooling for Adults In Even Start Families: 
Core Service Participants and Non-Participants (1992-93 Program Year) 

Legend 

Core Participants 

NonpartlcJpants 
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Exhibit 4.14: Distribution of CASAS Functional Literacy Level 
for Entering Adults (1 992-93 NEIS Data Set) 

80% ' I 




B*Olnnlno Buk> tattrmadlat* Hfch School 

CASAS Functional Level 



Exhibit reads: 71 percent of Even Start adults had CASAS pretest scores that placed them in the "high school" 
functional level. 

Note: Based on data from 120 of 340 projects. 

primary language (214). Adults in families with higher incomes also tended to enter with 
higher literacy levels than adults in families with lower incomes. 

Exhibit 4.16 compares CASAS reading achievement pretest scores of Even Start adults 
with the pretest scores of adults in other related programs. In one evaluation of the 
"321" Program (an adult basic education program in California), researchers (CASAS, 
1992) reported pretest means of 217 scale score points for 5,029 ABE students and 
210 points for 31,006 ESL students, lower entry levels than their Even Start 
counterparts. The California programs served a more ethnically diverse population than 
Even Start, often with more limited English proficiency, and many participants enrolled to 
meet a state or federal mandate. One explanation for the lower scores among California's 
321 participants is that they required tests for adults whose first language was not 
English, whereas the Even Start evaluation did not include such a requirement, eliminating 
these potentially low scorers from the study: 

The California-funded GAIN (Greater Avenues for Independence) program for Aid to 
Families with Dependent Children applicants and recipients conducted literacy testing on 
all participants. An early study on this program reported an average pretest of 233 
(CASAS, 1990). The higher literacy level of GAIN adults was not surprising since more 
than half of the participants of this welfare reform program had high school diplomas, or 
other degrees before entering the program. 
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Exhibit 4.15 

Pretest Scores on the CASAS Reading Survey 
(Scale Score Points; NEIS Data Set) 


Group 


Number of 
Adults 


Pretest 
Mean 


Pretest 
Standard 
Deviation 


Highest grade at intake 








Grades 0-4 


54 


194.6 


15.2 


Grades 5-8 


723 


225.0 


16.5 


Grades 9-12 


3,030 


230.7 


12.9 


Diploma/GED 


667 


232.7 


13.2 


Age at intake 








16-20 


752 


231.2 


11.7 


21-25 


1,561 


231.2 


13.0 


26-30 


1,280 


229.2 


15.3 


31-35 


596 


227.9 


14.8 


35-40 


279 


226.3 


17.9 


Over 40 


174 


223.3 


17.2 


Primery language is English 








Yes 


3,948 


231.9 


12.1 


No 


596 


214.2 


18.3 


Family annual income 








Under $5,000 


2,076 


228.7 


13.6 


$ 5,000 - 10,000 


1,209 


229.1 


14.8 


$10,000 - 15,000 


557 


230.0 


15.6 


$15,000 - 20,000 


324 


231.6 


15.0 


$20,000 - 25,000 


171 


233.6 


14.1 


Over $25,000 


144 


237.1 


14.7 


TOTAL 


4,694 


229.6 


14.4 


Exhibit reads: Adults entering Even Start without a high school diploma or GED performed at their 
expected functional level on the CASAS pretest. 
Note: Based on reports from 120 of 340 projects. 
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Exhibit 4.18: Primary Language of Adults in Even 
Start Families (1992-93 Program Year) 



English 



66.2% 




Other 



Exhibit reads: 66.2 percent of Even Start adults reported English as their primary language. 
Note: Based on data from 270 of 340 projects. 
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Exhibit 4.1 9: Primary Language of Participating Adults 
in Even Start Families, by Program Year 

[74%l 



20% 



[15%. 



10% -j 

j- 

o%4- 



16% 



1969-90 



120% 



T6%: 



1990-91 



|22%[ 



[6%, 



1991-92 



English 




Spanish 

[28%] 



Other 



18% 



1992-93 



Exhibit reads: Spanish was the primary language of 1 5 percent adults in 1989-90. This percentage rose each 
year, to 26 percent by 1992-93. 

Note: Based on reports from 1 20 projects in 1989-90, 1990-91, and 1991-92, and from 270 of 340 projects 
in 1992-93. 
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Exhibit 4.21 presents information about the language used by adults when reading to their 
children. Seventy-one percent of Even Start adults for whom English was not the primary 
language reported that they read to their children in their primary language, 1 7 percent 
of this group read to their children in English, and 9 percent read in both English and their 
primary language. 



Employment Status of Participating Adults in Even Start Families 

Most Even Start adults were not employed at the start of the program year. Seventy-six 
percent of the adults who participated in Even Start core services were not employed, 1 6 
percent were employed full-time, and 8 percent were employed part-time (Exhibit 4.22). 
Exhibit 4.23 expands on this information by showing the duration of employment. Eighty 
percent of the adults who were not employed had been so for more than 12 months. 
Over half (51 percent) of the adults who were employed had been so for more than 12 
months. 



Characteristics of Participating Children 

This section of the report presents data on more than 20,000 children in families that 
participated in core Even Start services during the 1992-93 program year. 



Age of Participating Children 

Even Start projects focus somewhat on children in the middle of the eligible age range 
(birth through age 7): 51 percent are 3, 4, or 5 years of age; 37 percent are less than 
3, and 12 percent are 6 or 7 years old (Exhibit 4.24). As existing projects matured and 
recruited new families, and as new projects were funded, the average age of participating 
children dropped-from 4.3 years in 1989-90 to 3.7 years in 1992-93. Over this same 
period, the percentage of infants and toddlers in Even Start (less than 3 years old) 
increased from 27 percent to 37 percent, the percentage of preschoolers (ages 3 or 4) 
increased from 33 percent to 38 percent, and the percentage of school-age children (ages 
5, 6, or 7) decreased from 40 percent to 25 percent (see Exhibit 4.25). These trends 
could have occurred for several reasons. First, in 1991 the Even Start legislation was 
changed to allow children to participate starting at birth instead of at age 1 , as in the 
original law. Second, the Department of Education actively encouraged projects to extend 
services to young children, so that projects were serving the entire 0-8 age range. And 
finally, as projects developed and matured they became more comfortable extending 
services to young children, especially the siblings of children already in the program. 
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Reported English Language Facility of Adults Participating 
in Even Start Core Services, 
rur Muuiis wnosu riisi Language is imoi cngiisn 
(1992-93 Program Year) 




Reported English Language Facility 


Percent of 
Adults 


Speaks English 




Very well 


24% 


Somewhat 


51% 


Not at all 


25% 


Reads English 




Very well 


15% 


Somewhat 


48% 


Not at all 


37% 


Understands English 




Very well 


23% 


Somewhat 


55% 


Not at all 


22% 


Exhibit reads: Of adults whose first language is not English, 24 percent report speaking English very 
well. 

Note: Based on reports from 270 of 340 projects. 



Exhibit 4.21 




Language Used to Read to Child 
for Adults Whose First Language is Not English 
(1992-93 Program Year) 




Language Used to Read to Child 


Percent of 
Adults 


Primary language 


1 1 OL 
1 1 10 


bnglish 


1 / 10 


Both 


a o/ 
3 70 


Other 


3% 


Total adults 


100% 


Exhibit reads: Of adults whose first language is not English, 71 percent read to their child in their 
primary language. 

Note: Based on reports from 270 of 340 projects. 



Exhibit 4.22: Employment Status of Adults In Even 
Start Families (1992-93 Program Year) 
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Exhibit 4.23 




Duration of Employment Status of Adults in Even Start Families 
(1992-93 Program Year) 


Emnlovment Status/Duration 


Percent of 
Adults 


Employed 




Less than 6 months 


31% 


6 to 1 2 months 


18% 


More than 12 months 


51% 


Not employed 




Less than 6 months 


11% 


6 to 1 2 months 


9% 


More than 12 months 


80% 


Exhibit reads: Of the Even Start adults who were not employed, 80 percent had been so for rr.ore than 
1 2 months. 

Note: Based on reports from 270 of 340 projects. 



Exhibit 4.24: Age of Participating Children In Even Start Families 
(1992-93 Program Year) 
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Exhibit 4.25: Age of Participating 
Children, by Program year 



142*1 

S 



40% 



J40%i- 



O 1 



530% -> 



27% 



38%' 



3,4 years 

i38%: 



0. 1.2, years 



' ■ ■ ■ ■ .(30%) ' ' 
[27*1 - 



31*1 ' 



^[26%; 
5,6,7 years 



20%- 



1989-90 



1990-S1 



1991-92 



1992-93 



Exhibit reads: The percentage of Even Start children who are 0, 1 , or 2 years of age increased from 27 percent 
in 1989-90 to 37 percent in 1992-93. 

Note: Based on reports from 120 projects in 1989-90, 1990-91, and 1991-92, and 270 of 340 projects in 
1992-93. 



Gender of Participating Children 

The percentage of male and female participating children is shown in Exhibit 4.26: 50 
percent of The children were male and 50 percent were female. 



Race/Ethnicity of Children in Even Start Families 

Raciai and ethnic categories for children are presented in Exhibit 4.27 and, as expected, 
are similar to those presented for adults: 39 percent of children were identified as white, 
28 percent of children were African-American, 3 percent were Native American, and 8 
percent were Asian/Pacific Islander or other categories. For 22 percent of children, 
Hispanic was listed as their ethnic heritage. Of this group, 74 percent were Mexican, 1 1 
percent were Puerto Rican, and 15 percent were "other Hispanic" (see Exhibit 4.28). 



Educational Experiences of Participating Children 

Fifty-five percent of Even Start children were reported to have had no formal educational 
experience prior to the beginning of Even Start, 22 percent hnrj a preschool experience 
(either Head Stait or some other preschool), 1 1 percent had p. icipated in kindergarten, 
and 8 percent participated in a primary grade (Exhibit 4.29). 
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Exhibit 4.28: Ethnic Background of Hispanic Children in Even Stan Families 

(1992-93 Program Year) 



Other Hispanic 



Puerto Rlcan 




Exhibit reads: 74.1 percent of Hispanic children in Even Start were identified as being of Mexican background. 
Note: Based on reports from 270 of 340 projects. 



Exhibit 4.29: Previous Educational Experience of Children 
Participating In Even Start Core Services (1992-93 Program Year) 




HmdStart Olhar Ptmc*»oI KkvWgwwn Prlmvy School Ottxr Non* 



Exhibit reads: 12 percent of Even Start children were in Head Start prior to joining Even Start. 
Note: Based on reports from 270 of 340 projects. 
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Disabilities of Participating Children 



Of all children participating in Even Start core services, 7 percent were reported to have 
a disability. Exhibit 4.30 shows the specific types of disabilities, none of which pxist for 
more than about two percent-of the Even Start population. Learning problems were cited 
for 31 percent of the children with disabilities (2 percent of all Even Start children), 
emotional problems for 12 percent, hearing problems for 9 percent, and "other" physical 
disabilities for 9 percent. 



Vocabulary Skills of Even Start Children 

This evaluation used the Peabody Picture Vocabulary Test (PPVT) to assess children's 
receptive (hearing) vocabulary. PPVT scores are standardized, so that a score of 100 
represents the average score for the norms group at each age level. Across children of 
all ages, pretest scores averaged 79.8 standard score points (Exhibit 4.31). This 
corresponds to the 9th percentile when compared to national norms and points out the 
low verbal skills of children prior to entry into Even Start. 



Comparison With Head Start and CCDP 

Even Start is a comparatively new federal program and it is of interest to see how the 
characteristics of Even Start participants compare to the characteristics of participants 
in other similar federal programs. Such cross-program comparisons are never easy 
because measures are often not comparable across studies. Nevertheless, Exhibit 4.32 
presents selected data on Even Start families, on families that participated in Head Start, 
and on families who are participating in the Comprehensive Child Development Program 
(a family support program funded by the U.S. Department of Health and Human Services) . 

Even Start families are less often headed by a single parent than families who participate 
either in Head Start or in CCDP (37 percent single parent families in Even Start vs. 55 



Exhibit 4.30 




Types of Disabilities for Children Participating in Even Start Core Services 

(1992-93 Program Year) 


Type of Disability 


% nf Children 
with Disability 
(N= 1,589) 


% nf Hhilrirpn 

/U \Jl 1 1 1 1 vJ 1 CI 1 

in Even Start 
(N = 22,429) 


Specific learning problem 


31% 


2.3% 


Emotional problem 


12% 


0.9% 


Hearing problem 


9% 


0.6% 


Other physical disability 


9% 


0.6% 


Visual problem 


6% 


0.5% 


Mental retardation 


5% 


0.4% 


Speech problem 


1% 


0.0% 


Orthopedic problem 


3% 


0.2% 


Other 


24% 


1.8% 


Exhibit reads: Six percent of Even Start children who had 
represents 0.5 percent of all Even Start children. 
Note: Based on reports from 270 of 340 projects. 


a disability have a visual problem. This 



Characteristics of Limited English Proficient Adults 

Some Even Start projects focus almost exclusively on adults who have limited English 
proficiency (LEP), while other projects serve mixed populations. For this evaluation, 

familips with limitpri Fnnlich nrnf 'm'lonr'Kt \moro HofinoH ue fomilioe fnr m,h/-»m the «irnot 







Exhibit 4.31 










PPVT Pretest Scores 
(Standard Scores from the NEIS Data Set) 






Group 


1 


N 


Mean |j 


SD 


Age at pretest 












3-0 to 3-1 1 






778 


81.3 


15.4 


4-0 to 4-1 1 






1,134 


76.3 


17.5 


5-0 to 5-1 1 






443 


81.4 


16.6 


6-0 to 6-1 1 






225 


86.4 


17.4 


7-0 to 7-11 






89 


85.0 


15.7 


Gender 












Male 






1,305 


79.4 


17.1 


Female 






1,292 


80.4 


16.8 


Ethnic background 












Asian 






48 


75.9 


17.7 


African-American 






1,018 


74.4 


16.3 


Hispanic 






378 


74.2 


17.4 


Native American 






108 


84.3 


13.9 


White 






1,094 


86.5 


15.2 


Prior preschool experience 










No 






1,781 


78.9 


16.8 


Yes 






898 


81.5 


17.2 


Highest grade attained by target parent 








Grade 0-4 






20 


68.8 


16.1 


Grade 5-8 






378 


77.9 


17.2 


Grade 9-12 






1,668 


79.8 


16.9 


Diploma or GED 






521 


82.0 


16.6 


Total 






2.677 


79.8 


17.0 


Exhibit reads: The everege PPVT pretest score for moles wes 79.4 stenderd score point6. 
Note: Besed on reports from 120 of 340 projects. 
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Exhibit 4.33 



Intake Characteristics of Adults Who Enter 
Even Start with Limited English Proficiency 



Intake Characteristics 



Percent of Adults 



Educated outside of the U.S. 
Did not reach 9th grade 
Single parent 
Unemployed 

Annual income < $15,000 

Primary source of income is government 
assistance 



K"5 



86% 
60% 
18% 
78% 
83% 
48% 



o 
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Cohort 1 and 2 projects, aggregated to the project level, produced findings in three fairly 
non-overlapping areas: category of need, age of children, and ethnic background. 

Category of need. Most projects target families in which the primary caregiver does not 
have a high school diploma or a GED and is on welfare or has an income below the 
poverty level. Two special categories of projects can be identified: 
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Steps Used in Formal Screening 



Projects were asked to identify the formal steps they used to screen participants by 
responding to a checklist and by writing in additional steps or activities. A summary of 
responses is presented in Exhibit 5.2. Nearly all of the projects verified the eligibility of 
potential participants (93 percent) and gave a basic orientation (88 percent) during the 
screening process. About three-quarters of the projects assessed the basic skills of adults 
(76 percent) while 60 percent contacted other agencies as part of the formal screening. 
Half of the projects (49 percent) tested children as part of screening, and 31 percent 
provided some counseling during screening. 



Core Services Delivered: Types and Providers 

Core services may be provided by staff funded by Even Start or by staff from cooperating 
agencies (e.g., a local Head Start program). Consequently, Even Start project directors 
were asked to report the types of core services provided to Even Start participants by 
staff funded through Even Start, by staff supported by cooperating agencies, or by both 
Even Start staff find cooperating agency staff. 



Parenting Education Services 

Even Start projects delivered a wide range of services to help parents raise their children. 
More than 90 percent of the Even Start projects offered the following types of parenting 
education services (Exhibit 5.3): helped families nake use of services provided by other 
social service agencies, discussed parents' role in the education of their children, oriented 
parents and children to school routines, taught parents about child development, trained 
parents in child behavior management, worked on building parents' self-esteem, and 
instructed parents in life skills and in principles of health and nutrition. 

Depending on the specific type of parenting education service, 24 to 44 percent of the 
projects provided services exclusively through Even Start staff; 39 to 57 percent of the 
projects delivered services jointly by Even Start staff and by staff from cooperating 



Exhibit 5.1 

Successful Strategies for Recruiting Eligible Participants: 
(1992-93 Program Year) 


Rar*ri sit m fin t Qtrsit onv 
ngvi ui ii i ioi it. o ii a toy y 


Pa rpant rxf Drniapfo 


Public school referrals 


62% 


A r 1 1 (III f> i i . i t ' * 1 

Agency referrals (e.g., WIC, health clinic) 


58% 


Telephone contact 


50% 


Home visits to potential participants 


47% 


Word of mouth 


42% 


Referrals from Head Start staff 


34% 


Targeted mailings 


27% 


Mass media 


23% 


Joint efforts with collaborative agency 


9% 


Posters / flyers 


6% 


Presentations / visits to community agencies 


6% 


General result of greater community visibility 


5% 


Even Start recruiting program 


2% 


(open house, fun fair, etc.) 




Exhibit reads: 47 percent of the reporting projects identified home visits as a successful recruitment 
strategy. 

Note: Based on reports from 291 of 340 projects. 



Exhibit 5.2 




Steps Included in Formal Screening of Potential Participants 
(1992-93 Program Year) 


Screening Activity 


Percent of Projects 


Verify eligibility 


93% 


Orientation 


88% 


Assess basic skills of adults 


76% 


Contact other agencies 


60% 


Test children 


49% 


Counseling 


31% 


None 


1% 


Exhibit reads: 93 percent of reporting projects verified eligibility as a step in formal screening of 
potential participants. 

Note: Based on reports from 291 of 340 projects. 
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Exhibit 5.3: Types of Parenting Education Services Reported 
(1992-93 Program Year) 
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Self Eatosm 
Rote In Education 
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57.4% 
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149.3% 




26% 60% 

Percentage of Projects 



76% 



100% 
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Ex ibit reads: About 40 percent of the Even Start projects used their own resources to provide parenting 
education in life skills. 

Note: Based on reports from 298 of 340 projects. 



Exhibit 5.4: Types of Adult Education Services Reported 
(1992-93 Program Year) 
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As would be expected, almost all Head Start and Chapter 1 pre-K services were provided 
by cooperating agencies, as were most kindergarten and primary school services. About 
30 percent of the projects provided "other preschool" services directly by Even Start 
staff. This reliance on existing providers is not surprising given the high cost of early 
childhood education services and their widespread availability through cooperating 
agencies and the public schools. 

Thus, Even Start projects are most likely to participate in the direct provision of services 
for parenting education and are more likely to delegate provision of services for adult 
education and for early childhood education. This fits with Even Start's mandate to build 
on existing services. In most communities, programs for early childhood education and 
of adult basic education already exist, and Even Start projects are taking advantage of 



Exhibit 5.6: Types of Adutt/Child Services Reported 
(1 892-93 Program Year) 
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Exhibit read;.: Few adult/child services were provided by a cooperating agency. 
Note: Based on reports from 283 of 340 projects. 

patterns, such as special month-long courses during the summer. Exhibit 5.7 shows that 
64 percent of a!! projects provided parenting education year-round while 23 percent 
offered it only for the regular school year; 59 percent of all projects provided adult 
education year-round, while 29 percent offered it for the regular school year; and 55 
percent of all projects provided early childhood education year-round, while 27 percent 
offered it for the regular school year. 



Support Services: Types and Providers 

As defined for this study, support services are activities provided directly to Even Start 
families to enable them to participate in core services. Support services remove barriers 
that, if unattended, restrict a family's ability to receive instructional and educational 
services. Such activities as staff development and training, while enabling the project to 
provide effective services to its clients, are not considered support services because 
families are not the direct recipients. 



To avoid duplication, Even Start projects are expected to obtain support services from 
existing sources as much as possible. As is seen in Exhibit 5.8, more than 80 percent of 



Exhibit 5.7 




Term of Year Core Services Are Offered 
(1992-93 Program Year) 




L»ore service / lerm ot Tear 


Percent of 

Prnio o't o 
r 1 UJtJOlo 


Parenting education 




Year-round 


64% 


Regular school year 


23% 


Other 


1 1 % 


Adult education 




Year-round 


59% 


Regular school year 


29% 


Other 


13% 


Early childhood education 




Year-round 


55% 


Regular school year 


29% 


Other 


16% 


Exhibit reads: Sixty-four percent of all reporting projects provide year-round parenting education. 
Note: Based on reports from 273 of 340 projects. 



other support services were provided by at least half of the projects including health care 
assistance; referrals for employment, mental health services, child protective services and 
domestic violence; treatment for chemical dependency; and referrals for services needed 
by persons with disabilities. 

Across all types of support services, an average of 20 percent of Even Start projects 
provided the service directly, cooperating agencies provided support services in 17 
percent of the projects, and Even Start and cooperating agencies jointly provided another 
32 percent of projects. However, as might be expected, there is substantial variation in 
the extent to which different support services are provided by Even Start or by 
cooperating agencies. The percentage of Even Start projects providing support services 
with their own funds ranged from less than 10 percent for health care to 40 percent for 
child care. These findings suggest that Even Start projects did, as planned, obtain many 
support services from existing providers and stepped in to provide services not available 
locally. 
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Exhibit 5.8 

Percentage of Projects Providing Support Services and 
Other Special Activities Through Even Start and/or Cooperating Agencies 

(1992-93 Program Year) 


Support Services 


Even Start 


Both 


Cooperating 
Agency 


Neither 


Transportation 


34% 


47% 


9% 


10% 


Family advocacy 


30% 


50% 


5% 


15% 


Counseling 


19% 


43% 


22% 


16% 


Nutrition 


14% 


47% 


22% 


18% 


Child earn 


40% 


34% 


8% 


18% 


Meals 


23% 


33% 


25% 


19% 


Health care 


7% 


30% 


40% 


23% 


Employment 


19% 


42% 


12% 


27% 


Mental health 


24% 


36% 


12% 


28% 


Child protective services 


10% 


21 % 


32% 


37% 


Domestic violence 


17% 


31% 


12% 


40% 


Personal assistance 


11% 


21% 


23% 


45% 


Chemical dependency 


18% 


23% 


13% 


46% 


Translators 


20% 


15% 


9% 


56% 


Parent stipend 


10% 


5% 


15% 


70% 


Exhibit reads: Most projects provided transportation services: 34 percent of the projects provided 
transportation exclusively through Even Start resources, 47 percent through Even Start and cooperating 
agency resources, and 9 percent exclusively through cooperating agency resources. 
Note: Based on reports from 297 of 340 projects. 



Special Activities 

In addition to providing core and support services, Even Start projects offer other 
occasional or one-time activities for the families they serve. These special activities are 
used to recruit families, provide information or training, celebrate participant 
accomplishments, and promote family pride, unity and sense of belonging. Exhibit 5.9 
lists several types of special activities. The categories are based on a content analysis 
of the written responses of projects in all cohorts. Because projects' responses were 
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Exhibit 5.10 



Number and Percentage of Arrangements to Provide Core Services by Type of Organization 

(1992-93 Program Year) 



Type of Organization 


Parenting 
Education 


Adult 
Education 


Early Childhood 
Education 


Total 


N 


% 


N 


% 


N 


% 


N 



Other departments/programs 627 25% 401 24% 622 33% 1650 

within public schools 
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for parenting education, 27 percent for adult education, and 31 percent for early 
childhood education. 3 

A wide variety of organizations cooperated with Even Start projects. The most common 
was "other departments and programs within the public schools" which accounted for 25 
percent of the parenting education arrangements, 24 percent of the adult basic education 
arrangements, and 33 percent of the early childhood education arrangements. The next 
most common type of cooperating agency was "local, county, state, or tribal agencies 
or organizations" which accounted for 23 percent of parenting education arrangements, 
21 percent of adult basic education arrangements, and 15 percent of early childhood 
education arrangements. Other cooperating agencies included postsecondary institutions, 
Head Start, day care or other preschool programs, foundations, volunteer groups, and 
other community-based organizations. Although they were mentioned infrequently, 
religious institutions (church, temple, or mosque) were involved in more than 80 
cooperative arrangements. 

Exhibit 5.1 1 displays the percentage of cooperative arrangements by core service area 
and source of authority over activities. The iocus of authority for activities is split 
between Even Start (about one-quarter of the agreements), the cooperating agency (about 
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Exhibit 5.11 






Percentage of Cooperative Arrangements to Provide Core Services 
by Core Service Area and Source of Authority Over Activities 
(1992-93 Program Year) 


Source of Authority Over 
Even Start Activities 


Parenting 
Education 


Adult 
Education 


Early Childhood 
Education 


Even Start 


25% 


26% 


22% 


Cooperating Agency 


46% 


48% 


53% 


Co-Deciding 


29% 


26% 


25% 


Exhibit reads: Twenty-five percent of all cooperative arrangements to provide parenting education were 
governed by Even Start. 
J Note: Based on reports from 297 of 340 projects. 



Cooperative Arrangements for Support Services 

Support services enable families to participate in Even Start core services by removing 
barriers to their participation. The support services most commonly provided through 
collaborative arrangements were transportation, meals, health care, counseling, and child 
care. Exhibit 5.13 shows that projects engaged in 2,027 cooperative arrangements for 
support services, and displays the number and percentage of projects providing a 
particular support service through a cooperative arrangement. 



Exhibit 5.13 

Cooperative Arrangements for Support Services 
(1992-93 Program Year) 


Number of Cooperative 
L___ Support Service Arrangements 


Percent of Projects 


Counseling 


218 


58% 


Transportation 


208 


49% 


Health care 


219 


46% 


Nutrition 


163 


38% 


Meals 


166 


37% 


Childcare 


188 


36% 


Advocacy 


178 


36% 


Employment referrals 


125 


29% 


Child protective services 


105 


25% 


Mental health 


109 


23% 


Referrals for battered women 


93 


21% 


Handicapped care 


81 


18% 


Chemical dependency referrals 


78 


15% 


Translators 


65 


14% 


Parent stipend 


41 


10% 


Total 


2,027 


100% 


Exhibit reads: Forty-nine percent of all reporting projects used cooperative arrangements to provide 
transportation. 

Note: Based on reports from 297 of 340 projects. 
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Implementation Problems and Solutions 



Two types of implementation issues are addressed in this section. First, projects were 
asked an open-ended question about major barriers to the implementation of Even Start 
as well as the strategies or solutions used to deal with the barriers. Second, projects 
were asked to identify features of the Even Start law or regulations which they felt 
needed revision to permit more effective implementation. 



Barriers to Program Implementation 

Many different types of barriers were identified by the 120 Cohort 1 and 2 Even Start 
projects for the 1992-93 program year (Exhibit 5.14). The most common barriers were 
difficulties in the recruitment, retention, attendance, and motivation of families (33 
projects across both cohorts), problems of communication and coordination with 
cooperating agencies (28 projects), a lack of transportation for families (25 projects), the 
difficult social service needs of the family and community (14 projects), a lack of quality 
child care (14 projects), financial problems (14 projects), staffing problems (13 projects), 
problems with facilities and space (12 projects), and problems with the evaluation (12 
projects). 



Features of the Law or Regulations that Would Enhance Implementation 

Exhibit 5.15 summarizes the responses given when the 120 projects in Cohorts 1 and 2 

were asked about features of the Even Start law or regulations that could be revised in 

order to enhance program implementation. A companion exhibit (Exhibit 5.16) lists 

specific comments from projects. The responses are paraphrased to give the reader a 

quick sense of what, in some cases, were fairly detailed points that projects wished to 

make. 

During 1992-93, 20 of the Cohort 1 and 2 projects responding to this item indicated a 
need to make eligibility criteria more flexible. This is a great reduction from 1990-91 
levels for this item. Twenty-four projects worried about the four-year funding cycle -- a 
clear indication of the fact that Cohort 1 projects were in the last year of their grant. 
Eight projects felt that the law should allow more flexible program design; 9 projects 



Exhibit 5.14 



Barriers 



Problems of Recruitment, 
Retention. Motivation, and 
Attendance 

Number of references: 33 



Project implementation Barriers and Resolutions 
(1992-93 Program Year) 



Resolutions 



Awards, family-oriented trips, made parents aware of being committed. 
Project kept a waiting list to fill vacancies quickly. 

Used individual, informal counseling sessions; gave referrals to counseling. 

Celebrated small successes along the way; broke goals into smaller parts. 

Involved the community involved in dealing with barriers by sending out newsletters. 

Better screening re: program components, e.g., understand this is more than adult education. 

Personal, face-to-face recruitment. 

"Bonus bucks," development of trusting relationship, more home visits. 
Use of various means of recruitment: word of mouth, home visit, and school activities. 
Attempted to provide alternative places for visits, e.g., library, public park, etc. 
Attempt to encourage involvement of entire family. 

Adherence to attendance policy; reentry of problem families is probationary. 
Even Start staff take adults to classroom and work with them. 
Incentives given (e.g., coupons for restaurants) to encourage attendance. 
Provided broader program to include more goals (e.g., drivers' education). 
Instituted attendance policy; participants were dropped for missing three sessions per month. 
Started volunteer-in-schools program to build familiarity. 
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Exhibit 5.14 
(continued) 

Project Implementation Barriers and Resolutions 
(1992-93 Proqram Year) 


Barriers 


Resolutions 


Transportation for Families 
Number of references: 25 


Began second van route, pay mileage to parents who drive. 
Two 12-passenger vans were purchased. 

Transportation is provided by Head Start, Even Start van, and carpooling. 
Still unresolved. 

Unresolved due to insurance concerns. Rural projects will face this barrier. 

Carpools; hold activities within walking distance when possible. 

Researching feasibility of providing transportation to site. 

Need for transportation decreased by holding adult education at Even Start site. 

Provided a shorter day so school buses could be worked out. 

Head Start, Friends of Family committed vans for use during 93-94 year. 

Collaborating agency provided funding for the purchase of a bus. 

Contracted with co-located daycare to provide some transportation. 

Work with transportation companies and looking into future rural transportation system. 

City Schools has provided a bus to ease the transportation burden. 

Limited van pick-up to two sites. 

We have had to double up and make two routes, sometimes running class late. 
Coordinating services with the district's Transportation Department. 
Home-based model. 

Reimbursing parents and volunteers for mileage lessens some difficulties. 
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Exhibit 5.14 




Exhibit 5.14 
(continued) 



Project Implementation Barriers and Resolutions 
(1992-93 Program Year) 



Barriers 



Financial Problems 
Number of References: 14 



Staff/Staffing Problems 
Number of references: 13 



Facilities/Space / Equipment 
Problems 

Number of references: 12 



O ?ven Start 



Resolutions 



Continued to seek outside resources to fund program expansion. 
Tied in with regular AE programs as much as possible. 
Resourcefulness, creativity with regards to public solicitation. 

Funding will only get worse. . 
Present to school boards; applied for grants; conducted a toy/book drive; contacted ICC members. 
Contacted parents on the waiting list as soon as a slot was available. 

Waiting list establishes ongoing dialogue with the school district about the future of program. 
Chosen to apply for a new Even Start grant which will allow us to expand. 
In-kind employee contributions, donations from community and collaborating agencies. 



Specific training in team building and group process. 

Immediate problems resolved: good staff retained; less suitable staff left. 

Talked and supported each other; regular breaks; inservice training. 

Tried to implement a new model that provided more integrated activity. 

Called in more volunteers. . 

The Even Start budget will be increased to allow teachers more time in the class. 

The organization that provides teachers will try harder to get committed teachers. 



PECO grant received for new building; to be completed fall 1994. 
Explored possibility of expanding to another school site. 
Still working on this. Moved once and acquired more space. 
Looked for a new space; hope to be in a different facility by fall. 
Rotated rooms; used outdoors when possible. 
Not really resolved, but adapted to the situation. 

Additional space provided by school district allows parent/child interaction. 
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(continued) | 




Project Implementation Barriers and Resolutions | 




(1992-93 Program Year) 


| Barriers 


I Resolutions j 



NEIS/Evaluation Problems 



Completed intake forms during orientation instead of home visits. 

Less paperwork will be required, right? 

We still use the Denver Developmental Screening. 







Exhibit 5.14 
(continued) 



Project Implementation Barriers and Resolutions 

(1992-93 Program Yea r) 

Barriers Resolutions 

Great Need/Number of Recruited volunteers to assist in classes; new collaborative argrement. 

clients Sought outside sources for various workshops, etc. 

In-kind employee contributions, donations from community and collaborating agencies. 
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Exhibit 5.14 
(continued) 

Project Implementation Barriers and Resolutions 
(1992-93 Program Year) 


Barriers 


Resolutions 


Lack of Support Services 
Number of references: 2 


Met with supervisors to discuss these issues and work together to address them. 


Complexity of Program 

Number of references: 1 


Asked parents to view group sessions before committing. 


Timing of Grant Award 

Number of references: 1 


Under state administration, fiscal year is changed to July 1 to June 30. 



Note: Based on reports from projects in Cohorts 1 and 2. 
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Exhibit 5.15 



Features of the Even Start Law or Regulations That May Need 
to be Revised to Permit More Effective Implementation 



Features of the Law 



Eligibility criteria 
Program design 
General evaluation concerns 
NEIS forms/instruments 
Four-year funding cycle 
Timing of fiscal year 
Other fiscal issues 
Concerns about state control 
Administrative concerns 
Other 



Number of Concerns 



1990-91 



73 
12 
25 
12 
1 
8 
6 
1 

15 
11 



1991-92 



25 
9 
13 
14 
12 
1 
4 
4 
2 
3 



1992-93 



20 
8 
9 
9 

24 
0 
4 
7 
2 
2 



Exhibit reads: In 1990-91 , 73 project directors identified eligibility criteria as a feature of the law that needed to be revised. 
Note: Based on reports from projects in Cohorts 1 and 2. 
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Exhibit 5.16 
(continued) 



Abstracted Project Comments on Features of the Even Start Law 
or Regulations That May Warrant Revision 

(1992-93 Program Year) 



I Features of the Law 


Commentary 


Program Design 


Clarify the number of families expected to be served at each project. 

Eliminate NDN requirement. Evaluation process doesn t measure Even Start family outcomes. 

Head Start restrictions cause planning problems. 

Mandate that Even Start. Head Start, and Chapter 1 work together. 

Need to give attention to prenatal instruction. 

Reconsider what counts as core service. Our two main components are not core. 
Rural programs need special design to accommodate transportation, childcare. 
Wording in Chapter 1 reauthorization should suggest coordination with Even Start. 


General Evaluation 
Concerns 

I 


Clarify responsibility and extent of the "local" evaluation. 
Evaluation component doesn't help at local level. 

Outcome assessment should relate to project model of curriculum and instruction. 

Reporting periods should coincide with the school year. 

The evaluation design with pre- and post-tests for children is terrible. 
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Abstracted Project Comments on Features of the Even Start Law 
or Regulations That May Warrant Revision 
(1992-93 Program Year) 


Features of the Law || 


Commentary 


NEIS 

Forms/Instruments 


Collect data on ALL participants to reflect true level of family involvement. 

Correct the difference of difficulty between the post and prct?st of CASAS. 

Eliminate interviews and testing of parents and children. 

Excessive amount of time needed to collect NEIS data. 

The instruments used do not necessarily measure parent effectiveness. 

Conduct more in-depth testing since CASAS doesn't render adequate information. 

NEIS should not require information on all families in project. 

New assessment instruments for non-English speakers. 

Paperwork in May is overwhelming. 

Revision of the testing requirements (paperwork is quite extensive). 
Space on Form ll's did not allow for enough core service hours for children. 
Too much paperwork. 

Use Dunst Family Strength Assessment, Infant Monitoring Questionnaire, & Early Screening Inventory 
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(continued) 




Abstracted Project Comments on Features of the Even Start Lew 




or Regulations That May Warrant Revision 




(1992-93 Program Year) 


Features of the Law || 


Commentary 



Four- Year Funding 
Cycle 



Four-year grant does not allow sufficient time for implementation. 
Fourth year projects who have met the Even Start requirements should have funding continued. 
Find a way for projects in years 5-8 to continue to be funded. 
Allow continued federal funding of model or outstanding programs after four years. 
Allow continuity of established programs. More than four years of funding to successful programs. 
Allow funding for program past four years. 

Amend law to fund successful programs first, then new programs. 
Consider funding exemplary projects beyond the four-year cycle. 
Continuation grant to programs after initial four-year demonstration period. 
Continuation of funding would assure continuity in staffing. 
Continue funding for projects meeting goals and objectives. 

Even Start should be funded as categorical aid rather than as demonstration projects. 
Fifth year funding is needed to continue project. 
Four years of funding is inadequate. 

Four-year limitation on funding has negative impact on attitudes. 

Make funding on a permanent basis, and contingent on demonstrated success. 

Give projects more than four years to implement. 

Grant cycle needs to be longer than four years. 

Longer range funding would permit more effective implementation. 

Do not limit funding to four years. 

Ongoing funding: eliminate/revise the "seed money" notion. 
Projects in fourth year are unclear about second cycle of funding. 

Remove regulation forcing fourth year programs to compete with new programs for funding. 
States should be encouraged to fund projects after four years at the 40% match. 
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should be interpreted with caution. A priority listing would be better, giving all projects 
the chance to consider each potential area of need. Nonetheless, responses of projects 
to the open-ended item gives a rough indication of the prevalence of certain needs. The 
responses presented here are limited to those from Cohorts 3 and 4, as those projects are 
most likely to need assistance. 

Exhibit 5.17 shows that the two most frequent technical assistance needs identified by 
Cohort 3 and Cohort 4 projects were for assistance in reaching and recruiting "hard-to- 
reach" parents (62 percent) and for information on other Even Start projects (49 percent). 
About one-third of the projects requested technical assistance in each of several different 
areas including the evaluation; integration of core services; home visiting; interagency 
collaboration; and accessing resources, transportation and job training. One-quarter of 
the projects asked for assistance with cross-project sharing and cooperative relationships, 
and 18 percent wanted assistance with Spanish language literacy. 



Exhibit 5.17 




Areas of the Program for Which Technical Assistance Is Wanted 
(1992-93 Program Year) 


|| Percent of Projects 


Reaching hard-to-reach parents 


62% 


Information on other Even Start projects 


49% 


Evaluation 


35% 


Integrating core services 


34% 


Home visits 


32% 


Interagency collaboration 


32% 


Accessing resources/transportation/job training 


30% 


Cross project sharing/cooperative relationships 


24% 


Spanish language literacy 


18% 


Exhibit reads: 62 percent of Cohort 3 and 4 projects requested technical assistance on reading "hard- 
to-reach" parents. 

Note: Based on reports from projects in Cohorts 3 and 4. 



Staffing Even Start Projects 

Information about the staff involved in Even Start projects comes from two forms added 
to the NEIS for the 1992-93 data collection. One form was filled out by individual Even 
Start staff members and the other was completed by project directors. Even Start staff 
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Exhibit 5.18 




Job Categories of Even Start Staff 
(1992-93 Program Year) 


Job Category 


Percent of Projects' 


Average Number of Staff 
Per Project 


Project administrator 


87% 


1.7 


Early childhood education 






Coordinator 


40% 


1.3 


Instructor 


76% 


4.1 


Aide 


58% 


3.7 


Adult education 






Coordinator 


36% 


1.4 


Instructor 


68% 


3.3 


Aide 


22% 


2.3 


Parenting education 






Coordinator 


29% 


1.1 


Instructor 


58% 


3.2 


Aide 


27% 


3.2 


Family recruitment 


22% 


2.4 


Case management 


36% 


2.8 


Evaluation 


45% 


1.6 


Clerical 


66% 


1.4 


Support services 


43% 


3.4 


Exhibit reads: 76 percent of Even Start projects have early childhood education instructors, with an 
average of 4.1 instructors per project. 

'Based on 270 out of 340 projects reporting staff information; includes only staff paid by Even Start or 
the local match. 

b Reflects average number of staff among projects with staff in that job category. 
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Exhibit 5.19: Duties Performed by Even Start Project Administrators 
(1992-93 Program Year) 



ECE Coordination 
EC 5 Instruction 
AE Coordination 
AE Instruction 
PE Coordination 
PE Instruction 
Family Recruitment 
Case Management 
Evaluation 
Clerical 
Support Service* 
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Exhibit reads: 25.1 percent of project administrators have primary responsibility for coordination of early 
childhood education. 

Note: Based on 382 project administrators in 270 projects. 



As might be expected, teachers and coordinators are most likely to have job 
responsibilities providing direct instruction or supervision in Even Start core services 
(Exhibit 5.20). In some projects, they may be asked to assist with recruitment, case 
management, or evaluation activities. 



Although Even Start teachers play multiple roles within the project, it appears that the 
majority of teachers (78 percent) have instructional responsibilities in only one core 
service (Exhibit 5.21). Eighteen percent teach in two core services, with early child 



Exhibit 5.20: I >uties Performed by Even Start Teachers and Coordinators 
(1992-93 Program Year) 



Project Administration 
ECE Coordination 
ECE Instruction 
AE Coordination 
AE Instruction 
PE Coordination 
PE Instruction 
Family Rscruitmsnt 
Casa Managamsnt 
Evaluation 
Clarteal 
Support Sarvtoss 
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Exhibit reads: 44.6 percent of Even Start teachers and coordinators provide early childhood instruction as a 
primary responsibility. 

Note: Based on 1 784 Even Start teachers and coordinators in 270 projects. 



Exhibit 5.21 


Even Start Teachers with Multiple Instructional Responsibilities 
(1992-93 Program Year) 


Number of Instructional Areas 


Percent of Teachers 


One 


78% 


Two 




Adult education and parent 
education 


4% 


Early childhood and parent 
education 


10% 


Early childhood and adult 
education 


4% 


Three 


4% 


Exhibit reads: 4 percent of Even Start teachers provide instruction in both adult education and parenting 
education. 

Note: Based on 520 teachers in 270 projects. 
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Funding Sources 



Staff providing Even Start services may be paid by one of three funding sources: federal 
Even Start funds, local matching funds, or through collaborating agencies. As Exhibit 
5.22 shows, collaborating agencies cover the costs of 54 percent of the staff who 
provide Even Start services. 



Exhibit 5.22: Source of Funding for Staff Providing Even 
Start Services (1 992-93 Program Year) 



Even Start 




Local Match 



Exhibit reads: Federal Even Start dollars pay for 32.3 percent of staff providing Even Start services. 
Note: Based on 270 out of 340 projects. 

Looking more closely at the type of staff from collaborating agencies (Exhibit 5.23), we 
see that the largest proportion of collaborating agency staff provide dirsct instruction in 
early childhood education (53 percent) and adult education (60 percent). Collaborating 
agency staff are much less likely to provide parenting education. 

The prevalence of staff from collaborating agencies may be somewhat exaggerated by the 
fact that many staff may have limited involvement with Even Start. For example, an early 
childhood education teacher at a collaborating agency may have only one or two Even 
Start children in her classroom. Another difficulty with the percentages in Exhibit 5.22 
is that some staff from collaborating agencies may be double-counted in the local match. 

Restricting the funding sources to federal Even Start dollars and local matching funds, we 
find that in Cohort 1 and 2 projects, 62 percent of project staff are paid with federal 
dollars. Conversely, 38 percent of project staff are paid through the local matching funds. 
This breakdown was fairly consistent across job categories, with the exception of the 
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Exhibit 5.23 




Job Categories of Staff from Collaborating Agencies 
(1992-93 Program Year) 


Job Category 


Percent of Projects" 


Average Number of Staff 
Per Project b 


Early childhood education 






Administrator 


52% 


3.0 


Instructor 


53% 


8.2 


Aide 


46% 


7.1 


Adult education 






Administrator 


48% 


2.2 


Instructor 


60% 


5.3 


Aide 


24% 


2.8 


Parenting education 






Administrator 


31% 


2.3 


Instructor 


36% 


3.6 


Aide 


19% 


2.7 


Total 


64% 


30.1 


Exhibit reads: 52 percent of Even Start projects have an administrator in early childhood education from 

a collaborating agency. 

"Based on 270 projects reporting. 

b Reflects average number of staff among projects with collaborating agency staff in that job category. 



following job categories where more than 75 percent of staff were paid with federal Even 
Start funds: aide in early childhood education classroom; instructor, coordinator or aide 
in parenting education; case management; and program evaluation. 

In Cohort 3 and 4 projects, 1 5 percent of project staff are paid by federal Even Start 
dollars and 25 percent through the local match. The differences between the earlier and 
later cohorts reflect that a project's local match requirement increases over the four years 
of the project. 



Part-Time and Full-Time Status of Staff 

Overall, 23 percent of Even Start staff work at least 1200 hours per year for the project 
(Exhibit 5.24), which translates into about 35 hours per week for nine months and was 
considered full-time. Approximately 38 percent work part-time (defined as between 300 
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Exhibit 5.24 




Part-time and Full-time Status of Even Start Staff 
{1992-93 Program Year) 






Percent of Staff 


Job Category 


Limited Part-time 
(<300 hours/yr.) 


Part-time 
(300-1200 hours/yr.) 


Full-time 
O1200 hours/yr.) 


Project administrator 


31 % 


29% 


39% 


Early childhood 
education 








Coordinator 


ZO 10 


OD /o 


o» to 


Instructor 


ZO/o 


A O 0/ 


ol % 


Aide 


OK OA 
ZO to 


Ol 70 


25% 


Adult education 








Coordinator 


f Z /0 


000/ 

Zo7o 


oUto 


Instructor 


*fi /o 


OO70 


Zl 70 


Aide 


oU to 


A 0 0/ 


zy% 


Parenting education 








Coordinator 


£. o /o 


OO /o 


OS /o 


Instructor 


1f\0L 
ZU 70 


*tO 70 


Oil o/ 

o4% 


Aide 


14% 


45% 


41% 


Family recruitment 


19% 


35% 


46% 


Case management 


23% 


35% 


42% 


Evaluation 


46% 


27% 


27% 


Clerical 


35% 


39% 


26% 


Support services 


49% 


43% 


8% 


TOTAL 


38% 


38% 


23% 


Exhibit reads: 39 percent of project administrators work full-time for Even Start. 
Note: Based on 270 out of 340 projects. 



and 1 200 hours per year) and another 38 percent work on a limited part-time basis for the project 
(less than 300 hours a year). 



Full-time staff are more likely to provide early childhood or parenting education. In 
contrast, adult educators are more likely to work on a limited part-time basis for Even 
Start. Staff involved in evaluation, clerical work and support services work are also more 
likely to work for Even Start on a limited part-time basis. 
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As Exhibit 5.25 shows, more than half of the staff paid through the local match work on 
a limited part-time basis for Even Start. It is likely thai these individuals have additional 
responsibilities with the local matching agency (e.g., school district, community college) 
for the remainder of their time. Only 10 percent of staff paid through the local match 
work full-time for Even Start. In contrast, 31 percent of staff paid directly through the 
Even Start grant work full-time. 



Exhibit 5.25: Part-Time and Full-Time Status by Funding Source 
(1 992-93 Program Year) 



Full-time 




Staff Paid by Even Start Staff Paid Through 

Federal Funds Local Match 



Exhibit reads: 41.3 percent of staff paid by Even Start work full-time for the program, compared with 10.3 
percent of staff paid through the local match. 
Note: Based on 270 out of 340 projects. 



Using Volunteers 

Forty-four percent of Even Start projects use volunteers in the delivery of early childhood 
education (Exhibit 5.26y, with an average of 14 volunteers per project. Volunteers are 
less prevalent in adult education and parenting education. Approximately 27 percent of 
projects use volunteers for general program support. 

In response to open-ended questions about minimum qualifications for volunteers in the 
core service areas, project directors from Cohort 1 and 2 projects listed criteria related to 
personality traits, education and experience. Some projects require volunteers to have 
certain minimum qualifications, while others interpret the notion of "qualification" more 
loosely, suggesting in their responses that they look for certain skills or personal 
characteristics in potential volunteers but do not have minimum requirements. 
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Exhibit 5.26 




Use of Volunteers in Even Start Projects 
(1992-93 Program Year) 


Job Category 


Percent of Projects* 


Average niumoer ot 
Volunteers Per Project b 


Early childhood education 


44% 


14.0 


Adult education tutor 0 


28% 


4.9 


Adult education aide" 


25% 


4.3 


Parenting education 


31% 


9.0 


General support 


27% 


7.5 


Exhibit reads: 44 percent of Even Start projects use volunteers in early childhood education, with an 
average of 14 volunteers per project. 
•Based on 270 out of 340 projects. 

"Reflects average number of volunteers among projects using volunteers in that job category. 
"Provides all adult education; does not assist adult education instructor. 
"Provides tutorial or other support to adult education instructor. 
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For the most part, projects choose volunteers for parenting education based on their 
experience and education. Most frequently, projects seek "experts in their field" to 
volunteer in this area, inviting members of the community to teach parenting topics and 
recruiting speakers from local community agencies. A number of projects choose 
volunteers for parenting education based on personal traits, such as empathy, 
commitment, and desire to work with educationally disadvantaged adults. 



Staff Work Experience 

Even Start project administrators have an average of nearly 9 years of experience working 
in early childhood education programs (Exhibit 5.27). The work experience of 
administrators is lower in adult education programs (4.3 years) and parenting education 
programs (5.8 years). This suggests that Even Start directors are more likely to come 
from programs for children and parents than from direct experience in adult education 
programs. 

Teachers and coordinators of -^rly childhood education also have roughly 10 years of 
work experience in their field (Exhibit 5.27). Teachers have an average of 8 years of 
work experience and coordinators have an average of 1 1 years of experience in that field. 
Parenting education coordinators tend to have more experience in early childhood 
programs (9 years) than in parenting education programs (6 years). This also is true for 
parenting education instructors, who report an average of 7 years of experience in early 
childhood education and less than 4 years of experience in parenting education. 
However, it is important to keep in mind that staff have responsibilities in more than one 
core service. Thus, the coordinators of early childhood education also may take on the 
role of parenting coordinator. In addition, early childhood education programs have a 
longer tradition than parenting programs. 

Adult education coordinators have an average of 7 years of experience with adult 
education programs. Adult education teachers in Even Start have an average of 5 years 
experience in adult education programs, nearly the same amount of experience as they 
have in early childhood programs. 

Exhibit 5.27 includes individuals with no experience in a field, which has the potential to 
lower the average values. To look at work experience another way, Exhibits 5 28, 5.29 
and 5.30 depict the percentage of staff who have one to five years of experience, six to 
10 years of experience and more than 10 years of experience in their field. 

While giving the same general message as the average values, Exhibit 5.28 points cut 
more clearly the extensive experience in early childhood education among Even Start staff 
who prov.de services in that core service. Approximately 35 percent of project 
administrators and 45 percent of early childhood coordinators have more than 10 years 
of work experience in early childhood education. Teachers are more evenly divided 
among the work experience categories, with nearly 30 percent having more than 1 0 years 
of experience. As would be expected, most classroom aides have between one and five 
years of experience in the field. 
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Exhibit 5.27 






Average Years of Work Experience Among Even Start Staff by Content Area 

(1992-93 Pro-ram Year) 




Average Years of Work Experience 


Job Category 


Fnrlv Childhood 
Education 


Adult 
Education 


roici mi ly 

Education 


Project administrator 


8.8 


4.3 


5.8 


Early childhood education 








Coordinator 


11.3 


2.8 


5.5 


Instructor 


7.9 


1.4 


2.5 


Aide 


4.8 


0.9 


1.4 


Adult education 








Coordinator 


5.7 


6.9 


3.9 


Instructor 


4.6 


4.8 


2.2 


Aide 


3.8 


2.9 


2.2 


Parenting education 








Coordinator 


9.3 


4.8 


5.9 


Instructor 


6.7 


3.1 


3.9 


Aide 


4.5 


2.2 


2.8 


Exhibit reads: On average. Even Start project administrators have 8.8 years of experience in early 
childhood education. 

Note: Based on 270 out of 340 projects. Individual staff may appear in more than one job category. 



In contrast to early childhood education, staff in adult education are much less likely to 
have more than 10 years of work experience in their field. Only 25 percent of adult 
education coordinators and 11 percent of adult education teachers have more than 10 
years of work experience in adult education. Nearly half of the staff who provide direct 
instruction in adult education for Even Start have less than five years of experience. 
However, it is important to keep in mind that this information includes only staff who are 
employed by Even Start, not staff from collaborating agencies. This limits our ability to 
make inferences about the general level of experience of all staff who provide adult 
education to Even Start participants. 
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Exhibit 5.28 Number of Years Experience in Early Childhood Education (ECE) 
Among Even Start Staff (1992-93 Program Year) 




Exhibit reads: 
education. 
Note: Based 



35.3 percent of project administrators have more than 10 years of experience in early childhood 
on 270 out of 340 projects. 



Exhibit 5.29: Number of Years Experience in Adult Education 
Among Even Start Staff (1992-93 Program Year) 



100% 



80% 



60% 



40% 




20% I- 



Project Administrator Adult Coordinator Adult Teacher 



Adult Aide 



Exhibit reads: 25.9 percent of project administrators have more than 10 years of experience in adult education. 
Note: Based on 270 out of 340 projects. 
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projects are more likely to have advanced degrees than teachers in the other two core 
service areas. 

Exhibit 5.33 shows the content areas of degrees held by Even Start staff for individuals 
with at least an associate's degree. Staff may hold degrees in more than one content 
area (e.g., a bachelor's degree in elementary education and a master's degree in child 
development, or a major in elementary education and a minor in language arts). The most 
prevalent content areas focus on education of young children, including early childhood 
education and elementary education. In contrast, a small percentage of Even Start 
instructional staff have training in adult education, language arts or English as a second 
language. 

Taken together, these results indicate that Even Start staff are likely to have a college 
degree but not always in fields specific to instruction in literacy or adult education. This 
is likely to be due, in part, to the greater prevalence of degree programs in elementary and 
early childhood education. 



Inservice Training and Staff Development 

Even Start staff were asked to indicate the type and length of inservice training received 
between January and June of 1993. Examples of inservice training or staff development 
activities include formal course work, conferences relevant to Even Start, and staff 
seminars. It does not include general staff meetings or curriculum planning sessions. 

Among Cohort 1 and 2 projects, 82 percent of Even Start instructional staff report at 
least one training session during the six-month period. In about 40 percent of projects, 
all (100 percent) of the instructional staff have attended at least one training or staff 
development activity. 

On average across projects, staff report about 40 hours of inservice training, although 
there is a considerable range among projects in the average hours of inservice training. 
In approximately 5 percent of projects, staff report less than 1 0 hours of inservice or staff 
development activities. At the other extreme, staff in 5 percent of projects report more 
than 100 hours of inservice or staff development. 

Exhibit 5.34 presents the average number of hours of training and average number of 
inservice sessions by various job categories across all projects (Cohorts 1, 2, 3 and 4). 
Project administrators and coordinators tend to report the greatest number of sessions 
and largest number of hours of inservice training, with an average of approximately seven 
to 10 sessions over six months and approximately 65 total hours during that period. Staff 
involved in family recruitment also report similar staff development or inservice 
experiences. Aides in early childhood and adult education report about half that number 
of sessions and hours. 
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Exhibit 5.34 




Amount of Inservice Training Received by Even Start Staff 
(1992-93 Program Year) 


Job Category 


Average Number 
of Sessions 


Average Number 
of Total Hours 


Project administrator 


7.7 


63.5 


Early childhood education 






Coordinator 


8.2 


66.1 


Instructor 


6.8 


47.7 


Aide 


5.7 


46.7 


Adult education 






Coordinator 


7.5 


67.6 


Instructor 


5.9 


42.8 


Aide 


5.5 


46.6 


Parenting education 






Coordinator 


8.6 


63.7 


Instructor 


8.6 


56.5 


Aide 


7.9 


57.0 


Family recruitment 


8.8 


68.6 


Case management 


7.9 


57.3 


Evaluation 


7.7 


54.0 


Clerical 


4.0 


30.5 


Support services 


3.7 


31.9 


Exhibit reads: Project administrators, on average, attend 7.7 inservice training sessions a year for a 
total of 63.5 hours. 

Note: Based on 270 out of 340 projects. Individual staff may appear in more than one job category. 
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Exhibit 5.35: Providers of Inservice Training to Even Start Staff 
(1992-93 Program Year) 



Multiple 



Even Start Staff 



Other 



Private Consultant 




X Community College 



-College/University 



School District 



Exhibit reads: 24.1 percent of inservice hours are provided by private consultants. 
Note: Based on 270 out of 340 projects. 



Exhibit 5.36: Content Area of Inservice Training 
(1 992-93 Program Year) 




Exhibit reads: 39.2 percent of inservice training sessions for Even Start staff focus on early childhood 
education. 

Note: Based on 101 Cohort 1 and 2 projects. 
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training for the implementation of High/Scope and other preschool curricula. Even Start 
staff also attended workshops on home visiting, cooperative learning, special needs 
children, emergent literacy, children's literature, whole language, and the use of 
manipulatives with young children. Projects sent staff to the National Association for the 
Education of Young Children and related state and regional conferences. Evaluation 
issues were addressed in alternative assessment workshops and trainings for various 
testing instruments. Staff learned about social issues affecting children in workshops on 
child abuse, health issues, and multicultural topics. 

Specific topics in adult education included the GED certificate, math for adults, reading 
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Some responded to the question more generally by advising new projects to provide 
frequent professional development opportunities for staff, to encourage life-long learning 
for all staff, and to conduct a staff needs assessment to determine what kinds of 
professional development activities staff would find most beneficial. 

In addition to providing formal staff development activities, Cohort 1 and 2 projects 
reported that they provided for staff development in other ways. In response to an open- 
ended question which asked projects to describe informal staff development activities and 
comment on the usefulness of these activities, about one-third of the projects provide for 
staff development through regular staff meetings. Several projects described mentoring 
relationships among staff and other opportunities for staff to pursue their own 
professional development plans. A number of projects commented that Even Start staff 
could access training provided by the school district or other agencies. Staff also learned 
from each other by visiting or sharing ideas with other Even Start projects, sharing 
feedback and working as a team. A few projects provided for staff development through 
computer linkages with other projects, staff journal writing activities, and interaction with 
community agencies. 



Project Directors' Recommendations about Even Start Staff 

Guided by a series of three open-ended questions, each Cohort 1 or 2 project director was 
asked to offer staffing advice to new projects. Common themes addressed by project 
directors include the value of using a team approach to staffing, establishing clear job 
descriptions, allowing staff to play flexible roles, and hiring staff with a variety of personal 
characteristics and professional backgrounds. Projects generally found full-time staff 
more desirable than part-time staff. Directors also cited frequent, relevant professional 
development activities as a critical element in the success of the program. Specifically, 
projects described the following staffing principles: 

• Use a team approach to staffing. Projects commented that staff 
working in teams effectively supported the delivery of integrated 
services that characterize family literacy programs. Projects reported 
that a team approach to staffing worked well because it allowed for 
joint planning and the sharing of job responsibilities to best meet 
client needs. In addition, teams facilitated cooperation and 
reciprocity between the child and adult components to create a 
necessary linkage between family learners. 

Some projects reported they were implementing the team staffing 
model recommended by the National Center for Family Literacy (the 
Kenan Model), which includes teams of two teachers and an 
educational assistant; others used teams of one teacher and two 
aides. Depending on the needs of the target community, some 
projects added other members to the three-instructor model, such as 
a social worker, home nutritionist, or child care aide. 
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Establish clear job descriptions. Projects stressed the importance of 
establishing job descriptions that are clear but not too rigid. Job 
descriptions should reflect the needs of clients, including flexible 
schedules, and allow staff to share responsibility for the delivery of 
various services as needed. Many projects stressed the importance 
of staff responsiveness to the needs of the target community, 
implying that new projects may find it useful to complete a needs 
assessment prior to establishing job descriptions and hiring staff. 

Utilize staff in flexible roles; integrate responsibilities. In general, 
projects indicated that the more integrated the approach to staffing 
the better. For example, in many cases early childhood and adult 
education staff share responsibility for the parenting component and 
directors frequently encourage staff to understand the job 
responsibilities of their co-workers. Projects found it useful to have 
versatile, knowledgeable team players who can readily adapt and fill 
in for other members of the team when needed. 

Many projects highlighted the importance of hiring detail-oriented 
clerical staff; qualified, experienced instructional staff; an on-site, full- 
time director; and a social worker or case manager. Although few 
projects reported having case managers, recruiters, or social workers 
on staff, those that did indicated that these individuals were critical 
members of their staff teams. 

• Seek enthusiastic and sensitive staff. Projects emphasized the 
importance of hiring qualified, committed, well-trained staff who are 
sensitive to the needs of disadvantaged families, as well as culturally 
and linguistically representative of the population to be served. 
According to many projects, the personality and enthusiasm of staff 
were critical when it came to getting the program started and gaining 
the trust of participants. In addition to enthusiasm, projects 
recommended hiring flexible, creative, non-judgmental, and sincere 
staff members. 

« Hire staff with a variety of backgrounds. Because Even Start 
programs provide a variety of services and levels of interaction to a 
diverse population, projects advocated hiring staff of varied ages, 
races, backgrounds, and personality styles. Projects found that this 
practice of striving for a balanced group of staff enhanced the 
interdisciplinary nature of the family literacy program, facilitated 
collegial interactions among staff, and increased the likelihood that 
staff would really connect with participants. A mixture of 
paraprofessional and professional employees was generally cited as 
an arrangement that worked well. Some projects found that clients 
worked especially well with staff who had been hired from the 
community served by the project. 
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Employ qualified staff. Projects cautioned against hiring individuals 
who are not qualified for their jobs. For example, projects mentioned 
as problematic hiring staff who did not speak the primary language of 
participants, who did not possess the skills necessary to be 
successful in their jobs, or who lacked postsecondary education. 
These under-qualified employees required more supervision, training, 
and direction than staff who were qualified for their jobs when they 
were hired. At the same time, however, other projects stressed the 
importance of not limiting the job responsibilities of staff based solely 
on a lack of formal education or experience. The responses of some 
directors indicated that they had struggled with trying to find a 
balance between hiring staff reflective of the client population and 
staff with the skills, education, and experience necessary to do the 
job. 

Use full-time staff. When commenting about staffing arrangements 
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This evaluation has multiple measures of program participation. One measure comes from _ 

an annual interview with a family member, usually the mother, conducted by Even Start I 
staff. On the interview form, project staff indicate whether, during the year, the family 

was an active participant in each of the three core services. This judgment on the part _ 

of the project staff member provides a binary measure of participation in each core service I 
area for each family for each year of the evaluation. A second measure of program 

participation comes from monthly "contact logs" that are used by project staff to record _ 

the number of hours of participation each month by each family in each core service I 



Exhibit 6.1 : Number of Families Participating 
in Even Start Core Services, by Program Year and Cohort 




Exhibit reads: Cohort 1 projects served 2,461 families in 1989-90. 

Note: Based on reports from 270 of 340 projects. Data for 1991-92 do not include Cohort 3 families. Totals 
include families in migrant projects. 



year). Thus, the same projects, with the same level of resources, were able to serve 
more than twice as many families in their third and fourth years of operation as in their 
first year. 

The large increases in numbers of families served over time can be attributed to the 
projects' need to deal with normal implementation problems in the first year of program 
operations (e.g., time had to be spent defining the program, recruiting staff, setting up 
operations) and the general difficulty of starting up a new program. Once these problems 
were solved projects became more efficient, with the extra time and resources being 
devoted to recruiting and serving additional families. 

A similar pattern is seen for Cohort 2 projects. A total of 1 ,938 families (average of 44 
families per project) were served in Cohort 2 projects during 1990-91 (their first year of 
operations), and 3,997 families (average of 91 families per project— a 106 percent 
increase) were served in 1991-92. This seems to represent steady-state operations for 
Cohort 2, since participation was relatively unchanged in Cohort 2's third year of service 
(3,972 families, an average of 90 families per project). 
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Participation Rates For Each Core Service 



The Department of Education expects that all Even Start families will participate in each 
of the three core service areas during their time in the program. Exhibit 6.2 shows the 
percentage of families that participated in each core service area during the four years of 
study for the 120 projects in Cohorts 1 and 2. Almost all Even Start families had a child 
that participated in early childhood education during each year of the evaluation: 90 
percent participated in 1989-90, 97 percent in 1990-91 , and 98 percent in 1991-92 and 
in 1992-93. Participation rates for parenting education were a little lower: 88 percent 
of families had a participating adult in 1989-90, 94 percent in 1990-91, 93 percent in 

1 991- 92, and 95 percent in 1 992-93. At the beginning of Even Start, participation rates 
were lowest for adult education (54 percent in 1989-90). The Department of Education 
provided technical assistance to local projects and participation in adult education 
increased to 80 percent in 1990-91, xo 90 percent in 1991-92, and to 91 percent in 

1992- 93. The percentage of families that participated in all three core service areas 
increased over the four years of study, from 46 percent to 75 percent to 84 percent to 
86 percent. 

The relatively low participation rates for projects in Cohorts 3 and 4 (see Exhibit 6.3) 
reflect the new funding status of those projects. During 1 992-93, 88 percent of the Even 
Start families in these cohorts had a child who participated in early childhood education, 
73 percent had an adult who participated in parenting education, 79 percent had an adult 
who participated in adult education, and 52 percent participated in all three core services. 
Participation rates for Cohort 1 and 2 projects were similarly low during their first year of 
operation. 

Each year approximately 10 to 15 percent of Even Start families participate in one, but 
not both, of the adult-focused core service areas. Exhibit 6.4 contains detailed data on 
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Exhibit 6.2: Percent of Families Participating in Even 
Start Core Services, by Program Year, for Cohorts 1 and 2 
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Exhibits reads: 90 percent of Even Start families in Cohorts 1 and 2 had a child who participated in early 
childhood education during the 1 989-90 program year. 
Note: Based on reports from 120 projects. 



Exhibit 6.3: Percent of Families Participating in Even 
Start Core Services, 1992-93, for Cohorts 3 and 4 
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Exhibit 6.4: Number and Percent of Families 
Participating in Even Start Core Services (1980-93) 


Core Service 


Cohort 1 


Cohort 2 


Cohort 3 


Cohort 4 


Migrant 


Total 


1 989-90 Participation 














Adult education 


54% 










54% 


Parenting education 


88% 










88% 


Early childhood education 


90% 










90% 


Adult ed. or parenting ed. 


95% 










95% 


Adult ed. and parenting ed. 


46% 










46% 


All core services 


46% 










46% 


Total number of families 


n = 2,461 










n = 2,461 


1990-91 Participation 














Adult education 


80% 


81% 






100% 


80% 


Parenting education 


94% 


93% 






94% 


94% 


Early childhood education 


97% 


97% 






100% 


97% 


Adult ed. or parenting ed. 


99% 


96% 






100% 


98% 


Adult ed. and parenting ed. 


75% 


78% 






94% 


76% 


All core services 


74% 


77% 






94% 


75% 


Total number families 


n = 4,770 


n = 1,938 






n=18 


n = 6,726 


1991-92 Participation 














Adult education 


89% 


91% 


na 




84% 


90% 


Parenting education 


93% 


93% 


na 




99% 


93% 


Early childhood education 


98% 


98% 


na 




100% 


98% 


Adult ed. or parenting ed. 


98% 


98% 


na 




100% 


98% 


Adult ed. and parenting ed. 


85% 


86% 


na 




84% 


85% 


All core sen/ices 


84% 


84% 


na 




84% 


84% 


Total number of families 


n = 5,570 


n = 3,997 


na 




n = 134 


n = 9,701 


1 992-93 Participation 














Adult education 


90% 


91% 


82% 


72% 


64% 


85% 


Parenting education 


95% 


95% 


77% 


70% 


50% 


86% 


Early childhood education 


97% 


99% 


88% 


88% 


88% 


91% 


Adult ed. or parenting ed. 


99% 


99% 


na 


na 


na 


na 


Adult ed. and parenting ed. 


86% 


88% 


na 


na 


na 


na 


All core services 


85% 


87% 


57% 


43% 


36% 


72% 


Total number of families 


n = 5,377 


n = 3,972 


n = 4,863 


n = 1,672 


n = 371 


n=16,255 


Exhibit reads: 54 percent of all Even Start families had an adult who participated in adult education during the 1989-90 program year. 
Note: Based on reports from 270 of 340 projects, 
na: Data not available. 
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Start. Similar questions were asked in the In-Depth Study, where 40 percent of the 
parents responded that they had participated either in adult basic education, GED 
preparation, or English-as-a-second language programs during the past year. The higher 
pre-Even Start participation rate for the In-Depth Study group may occur because the In- 
Depth Study sites overrepresent Hispanics, who are more likely than English-speakers to 
enroll in English-as-a-second language programs. In any case, data from the NEIS show 
that in the third and fourth years of program operations, over 90 percent of Even Start 
adults participated in adult education. 6 



Parenting Education 

Prior to joining Even Start, few parents had been involved in parenting education. Parents 
were asked questions about previous participation in parenting education programs as part 
of the In-Depth Study but not as part of the NEIS. Eight percent of the parents in the In- 
Depth Study indicated that they had taken part in a parenting education program prior to 
Even Start. Data from the NEIS show that in each year of program operations, over 90 
percent of Even Start adults participated in parenting education. 



Early Childhood Education 

Determining the extent to which children would participate in early childhood education 
programs in the absence of Even Start is difficult, because Even Start serves children from 
birth to age eight, and early childhood education participation rates differ bv age. In spite 
of this problem, Even Start parents were asked about the prior formal educational 
experiences of their child. Their responses (across all ages of children eligible for Even 
Start) indicated that 64 percent of children in Even Start had no prior formal educational 
experience, 1 4 percent had participated in Head Start, 1 3 percent took part in some other 
preschool, 12 percent were in kindergarten, and 5 percent were in primary school. 

It is not possible to disaggregate the Even Start data by age of child, and so data from the 
national longitudinal study of Chapter 1 were used to provide additional information on 
this issue. According to Puma et al. (1993), 26 percent of Chapter 1 students had 
participated in Head Start, and 35 percent had participated in some other preschool, a 
total of 61 percent. It makes sense that these percentages are higher than those reported 
by Even Start parents, because the Chapter 1 data were reported for first-grade children, 
whose parents were able to reflect on their child's entire preschool experience. On the 
other hand, Even Start parents' reports of preschool experiences were limited by the age 
of the child— if a child was three years old at entry to Even Start, that child could not have 
participated in Head Start. Because of this problem, the Chapter 1 estimate of preschool 
participation is likely to be a better estimate of the typical early childhood education 



6 Families had to participate in at least one core service in order to be counted as a 
program participant. We know that some families were recruited but never participated, 
and these were not counted. 



Even Start 



130 



Participation in Even 
Start Core Services 



participation rate for a disadvantaged population than the statistics provided by Even Start 
parents. 

A recent analysis by the U.S. General Accounting Office (1 993) relies on Census data and 
provides estimates of preschool participation for poor and nonpoor three- and four-year- 
old children. The GAO concluded that 22 percent of three-year-old children and 49 
percent cf four-year-old children from poor families had participated in preschool. The 
percentages were higher (33 percent and 57 percent, respectively) for children from 
nonpoor families. 

The NEIS data set tells us that in each year of program operations over 90 percent of 
Even Start children participated in early childhood education, and so it is clear that 
regardless of the basis of comparison. Even Start has increased participation in early 
childhood education programs. 

To summarize. Even Start has achieved its goal of increasing participation in its three core 
service areas, from somewhere in the 30 to 40 percent range to over 90 percent for adult 
education, from about eight percent to over 90 percent for parenting education, and from 
40-60 percent to over 90 percent for early childhood education. 



Project-Level Variation in Participation Rates 

Participation rates are not uniform across Even Start projects. And, in fact, the cohort- 
level averages presented above mask much greater project-to-project variation. Exhibit 
6.5 is a distribution of project-level participation rates in all three core services. The 
shape of the distribution shows that many projects are able to engage a large percentage 
of their families in all three core services: 90 percent or more of the families participated 
in all core services for 32 percent of the projects. It also shows that all families 
participated in each core service in 7 percent of the projects, while less than 60 percent 
of the families participated in each core service area in 35 percent of the projects. Most 
of the projects in this latter group belong to Cohorts 3 and 4; only 6 percent of the 
projects in Cohorts 1 and 2 had less than 60 percent of their families in all three core 
services. 

Exhibits 6.6, 6.7 and 6.8 are distributions of project-level participation rates for each of 
the three core service areas. The distributions look roughly similar in shape, and are 
essentially bi-modal. That is, one group of projects serves a relatively low percentage of 
families (less than 60 percent) in each core service area, and another group serves 90 
percent or more of their families. 



Participation in Even 

Even Start 131 start Core Services 

; ;>.■/ 

9 

ERIC 

ummmmmim 



son 



75 20% 



Exhibit 6.5: Distribution of Participation Rates 
by Project for All Core Services (1 992-93) 
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Exhibit reads: In 7 percent of the projects, 100 percent of the families participated in all three core services. 
Note: Based on reports from 265 of 340 projects. 



Exhibit 6.6: Distribution of Participation Rates 
by Project for Adult Education (1992-93) 
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Exhibit reads: In 22 percent of the projects, 100 percent of the families participated in adult education. 
Note: Based on reports from 272 of 340 projects. 
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Multi-Year Participation 



Although Even Start projects are funded for four years, few families take part for that 
amount of time. This fits with reports from project directors who, in the early years of 
the evaluation, identified the recruitment, retention, and motivation of families as the 
most common barrier to effective program implementation. 

Fifty-three percent of the families that began Even Start in Cohort 1 projects during the 
1 989-90 year participated only in that first year, 24 percent participated in both the first 
and second program years, 13 percent participated in three years, and 10 percent 
participated in all four years of the grant (Exhibit 6.9). A similar pattern is emerging for 
Cohort 1 families who started their participation in Even Start during 1990-91 and 1991- 
92. On the other hand, Cohort 2 projects appear to be more successful at retaining 
families across years. Of all Cohort 2 families that began Even Start in 1990-91, 27 
percent participated for one year, 42 percent continued into a second year, and 31 
percent participated in three years. 

These percentages are informative, but they are limited in that they only identify a family 
as having participated or not in a given year. They tell us nothing about the amount of 
participation during that year. A more detailed look at length of participation is given by 
Exhibit 6.10 which draws on monthly contact log data from Cohorts 1 and 2 to show the 
number of months of participation over a 24-month period. The months of service do not 
have to be consecutive, so that a family who participates in core services during October, 
skips November and December, and participates again in January would have two months 
of participation. 

The exhibit shows that 25 percent of the families received some core services in three 
or fewer out of the total possible 24 months, 50 percent of the families participated in 
seven or fewer months, 69 percent participated in 12 or fewer months, and about four- 
fifths (79 percent) participated in 24 or fewer months. Conversely, 31 percent 
participated in more than 12 months. 



Reasons for Exiting from Even Start 

There are many reasons for turnover of families in Even Start (see Exhibit 6.11). Some 
of these are positive, e.g., a parent found a job and moved out of the project's catchment 
area, and some are negative, e.g., the family lost interest or the program didn't meet their 
needs. A reason for leaving was reported for 70 percent of the families that left the 
program, a substantially higher percentage of families than was observed in the national 
evaluation of adult education programs, where program staff were able to list a reason 
for exiting for only 38 percent of the families that left the program (Development 
Associates, 1993). Families that exited the program for unknown reasons may be 
different in important ways from families that exited for a known reason. For example, 
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Exhibit 6.9 

Years of Participation by Cohort and Year of Intake 







One 


Two 


Three 


Four 








Year 


Years 


Years 


Ye 


ars 


Total 


Cohort 


Intake 
















% 




Year 


N 


% 


N 


% 


N 


% 


N 




Cohort 1 


1989-90 


1,315 


53% 


586 


24% 


329 


13% 


231 


10% 


2,461 




1990-91 


2,001 


55% 


1,009 


27% 


650 


18% 






3,660 




1991-92 


1,858 


55% 


1,525 


45% 










3,383 



Cohort 2 1990-91 520 27% 807 42% 611 31% 1,938 
1991-92 1,377 53% 1,221 47% 2,598 



Exhibit roads: Among families entering Even Start at a Cohort 1 site during 1989-90, 53 percent participated only 
in that program year while 24 percent also participated during 1990-91, 13 percent perticipeted in 1991-92, and 
10 percent participated in all four program yeirs. 

Note: Based on reports from 120 of 340 projects. 
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Exhibit 6.11 






Reasons for Leaving Even Start 
(1992-93 Program Year) 




Reason for Leaving 


N 


Percentage of 
Families Giving A 


No reason given 


5,015 




Reason given 


11,532 




Comnlptpri Dlanned pdunational orooram or 

wUI 1 lUluluU fJ ' « 1 1 1 1 \j \m ^UUUQl IU 1 lul fJ ' UUI dill Ul 

met personal objectives 


2 756 


24% 


Moved from area or changed programs 


2,779 


9/1 O/ 
^M- /O 


Lack of interest, refused to participate 


1,553 


13% 


Family crisis 


1,387 


12% 


Not eligible based on local or national 
criteria 


1,177 


10% 


Conflicts, barriers to participation 


595 


5% 


Service unavailable or project closed 


295 


3% 


Other 


1,010 


9% 


Exhibit reads: 24 percent of the families which left Even Start did so because they successfully 
completed their planned educational program. 
Note: Based on reports from 272 of 340 projects. 



families which leave for unknown reasons may be more likely to move, to be dissatisfied 
with the project, or to be difficult-to-reach families than families that leave for a specific 
reason. 

Based on families where there was a reason for leaving, completion of the planned 
educational program or meeting personal objectives was listed for 24 percent of families 
that exited Even Start. This includes families where all members completed their 
educational program as well as families where parents obtained their GED or a full-time 
job, at which time the entire family exited the program. 

Moving out of Even Start's catchment area or changing programs was an equally common 
reason for leaving the program, listed for 24 percent of families. This large percentage 
of movers raises the issue of whether project directors are aware of and are using their 
option to continue serving families that have moved but are close enough to continue 



Thirteen percent of the families left Even Start because of a general lack of interest in tlvj 
program and a subsequent refusal to participate. Another 12 percent had a family crisis 
of one sort or another that prevented them from participating. Ten percent left the 
program because they became ineligible due to a change in the family situation, i.e., there 
was no longer an eligible child or adult in the family. This could be due to federal or local 
eligibility requirements. Five percent gave a variety of reasons which suggested personal 
or structural conflicts or barriers to continued participation, such as: medical reasons, 
work conflicts, pregnancy, scheduling conflicts, child care problems, and a lack of 
transportation. Finally, three percent were in projects that closed or left because a key 
service was unavailable. 



Amount of Core Services Received 

The contact logs compiled by Even Start staff from projects in Cohorts 1 and 2 record the 
amount of time families spent in each core service area on a monthly basis. The mean 
amount of service for each core service area is substantially higher than the median, 
indicating that some families receive very large amounts of service, while many more 
received relatively small amounts of service. We present both medians and means in the 
following discussion. 

Even Start staff recorded contact hours for one adult and one child in each family. Thus, 
the participation data described below represents a lower-bounds estimate of service 
hours for families where multiple child or multiple adults participated in Even Start core 
services. 



Total Hours of Service Received 

Exhibit 6.12 shows the amount of service received by families who entered Even Start 
in 1990-91. The average family in that cohort participated in 107 hours of adult 
education (median of 41 ), 58 hours of parenting education (median of 29), and 232 hours 
of early childhood education (median of 102). Although not shown in a table, the average 
amount of service received by families in the five In-Depth Study projects is close to the 
overall figures cited above: 123 hours of adult education (median of 63), 48 hours of 
parenting education (median of 34), and 266 hours of early childhood education (median 
of 162). 

There is great variation in the total amount of core services received by Even Start 
families. Exhibit 6.13 is a distribution of total hours of adult education. It shows that 
more than half of the adults who participated in Even Start received less than 50 hours 
of adult education, 15 percent received between 50 and 99 hours, 8 percent received 
between 100 and 149 hours, and so on. Small percentages of families received very 
large amounts of adult education, (e.g., 10 percent received 300 or more hours). 
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Exhibit 6.12 

Measures of Amount of Core Services for 1990-91 Cohort 



Core Service 



Total hours served 
Adult education 
Parenting education 
Early childhood education 



Total months served 

Adult education 
Parenting education 
Early childhood education 



Average hours per month 
Adult education 
Parenting education 
Early childhood education 



Percentile 



25% 



50% 
(Median) 



75% 



Mean 



SD 



14 
11 
21 



41 124 107.1 169.4 
29 67 57.8 93.2 

102 330 232.3 305.0 



3 
3 
3 



5 
6 
6 



12 
12 
12 



7.8 6.6 
8.3 6.8 
8.1 6.5 



4 
2 
5 



9 
4 
16 



18 
8 
39 



13.5 
6.5 
26.0 



13.9 
7.4 
26.7 



Exhibit reads: Even Start adults participated in an average of 107.1 total hours of adult education. The 

median adult participated in 41 hours. 

Note: Based on reports from 120 of 340 projects. 



A similar pattern can be seen for parenting education (Exhibit 6.14) and for early 
childhood education (Exhibit 6.15). Sixty-six percent of Even Start families received less 
families received less than 50 total hours, 1 1 percent received between 50 and 99 hours, 
and the remaining 51 percent received more than 100 total hours, ranging up to 600 or 
more hours of instruction. 

Additional analysis of contact log data for adult education shows that the average number 
of instructional hours increased steadily over three years of measurement, from 68 hours 
in 1990-91 to 91 hours in 1991-92, and again to 107 hours in 1992-93 (Exhibit 6.16). 
This pattern of increasing amounts of instruction over time holds for each instructional 
area within adult education. It appears that as Even Start projects mature, the amount 
of participation on the part of adults increases, either as a function of better attendance 
or increased retention. 
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Exhibit 6.13: Distribution of Total Hours of Adult Education 
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Total Hours of Adult Education 
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Exhibit reads: More than half (55 percent) of Even Start adults participated in less than 50 hours of adult 
education. 

Note: Based on reports from 120 of 340 projects. 
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Exhibit 6.14: Distribution of Total Hours of Parenting Education 



66% 



V40 




60-99 100-149 160-199 200-240 

Total Hours of Parenting Education 



260+ 



Exhibit reads: Two-thirds of Even Start adults participated in parenting education less than 50 hours. 
Note: Based on reports from 120 of 340 projects. 
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Exhibit 6.15: Distribution of Total Hours of Early Childhood Education 
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Exhibit reads: 38 percent of Even Start children participated in early childhood education less than 50 hours. 
Note: Based on reports from 120 of 340 projects. 




Exhibit reads: Even Start adults received an average of 107 hours of adult education instruction in 1992-93. 
Note: Based on reports from 120 of 340 projects. 
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To put these findings in perspective. Exhibit 6.17 contrasts the average number of hours 
spent by Even Start adults participating in different types of adult education with the 
amount of time spent by adults in adult education programs across the country as 
measured by the national evaluation of adult education programs (Development 
Associates, 1993). Across all three years of Even Start data collection. Even Start 
participants received more instruction, on average, than participants in regular adult 
education programs (93 hours vs. 80 hours). When broken down by instructional area, 
the Even Start vs. regular adult education difference is negligible for adult basic education 
and English-as-a-second-language programs, but is quite large for adult secondary 
education/GED preparation programs (average of 91 hours vs. 6b hours). 





Exhibit 6.17 




Average Adult Education Hours by Instructional Area, 
Even Start and Other Adult Education Programs 


Instructional Area 


Even Start 
(Average Hours)" 


Other Adult Education 
Programs 
(Average Hours) b 


Adult basic education 


79 


74 


Adult secondary 
education/GED preparation 


91 


63 


EngNsh-as-a-second 
language 


109 


107 


Total 


93 


80 


Exhibit reads: Even Start adults received an average of 93 hours of adult education. 
"Represents average of data from 1990-91, 1991-92, and 1992-93. 

"From National Evaluation of Adutt Education Programs, Development Associates (1993), p. 60. 



The differences favoring Even Start are much larger if we assume that the national adult 
education evaluation is measuring mature adult education programs, and that mature Even 
Start projects would provide instruction at levels similar to the most recent year of data 
collection (1992-93) shown in Exhibit 6.16. The 1992-93 data show that, across all 
instructional areas. Even Start participants received 34 percent more adult education 
instruction than do participants in regular adult education programs: 62 percent more for 
adult secondary education/GED preparation, 35 percent more for adult basic education. 



the typical Even Start family received 9 hours of adult education (mean of 13.5 hours), 
4 hours of parenting education (mean of 6.5 hours), and 16 hours of early childhood 
education (mean of 26 hours) per month (Exhibit 6.12). This could equate to a once-a- 
week adult education class for two hours per week and a once-a-week parenting 
education visit or class for one hour per week. 

As was the case for total hours of core services, there is great project-to-project variation 
in the average monthly hours of each core service. Exhibits 6.18, 6.19, and 6.20 show 
project-level distributions of the average hours per month of service for adult education, 
parenting education, and early childhood education. Eighty-five percent of the projects 
provide an average of fewer than 19 hours per month of adult education, with small 
numbers of projects providing higher amounts (Exhibit 6.18). The same pattern is seen 
for parenting education-94 percent of the projects provide an average of 14 or fewer 
hours of parenting education per month (Exhibit 6.19). There is even greater variation in 
the amount of early childhood education services received, and the distribution of average 
amount of service received (Exhibit 6.20) is fairly flat, except that 18 percent of the 
projects provide an average of 45 or more hours of early childhood education per month. 

Amount of Early Childhood Education. The amount of early childhood education service 
is related to the age of the child being served (Exhibit 6.21). The typical Even Start child 
who is less than one year of age received 6.3 hours per month of early childhood 
education (mean of 15.5 hours). This amount stays roughly constant for one-year-olds, 
and then increases to 1 0.6 hours per month (mean of 20.0) for two-year-olds, 20.3 hours 
(mean of 29.2) for three-year-olds, and 29.6 hours (mean of 35.5) for four-year-olds. The 
pattern breaks for five-year-old children, who receive fewer hours of early childhood 
education services through Even Start (median of 12.0 hours per month, mean of 25.2) 



Exhibit 6.18: Project Level Distribution of Average 
Hours Per Month of Instruction for Adult Education 
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Exhibit 6.23: Percent of Projects Using Different Amounts 
of Home-Based Service, by Core Servioe Area 




Adult Education Parenting Education Early Childhood Education 

Core Service Area 



Exhibit reads: 34.2 percent of the projects provided over two-thirds of parenting education services to adults 
and children together. 

Note: Based on reports from 1 20 of 340 projects. 

Effect of Home-Based Services on Retention and Service 
Amount 

Home-based services are an important part of Even Start projects, and the preceding 
discussion showed that there is substantial variation in the extent to which projects rely 
on this service delivery mode. Home-based services are seen as important in their own 
right-as being a useful way of delivering services directly to families. In addition, when 
Even Start staff travel to a family's home there is hope that the family will feel that Even 
Start staff value them, that family members will learn to trust Even Start staff and, in the 
long run, that family members will be willing to participate more fully in Even Start 
activities. 

Data from the NEIS show that there is a strong positive relationship between amount of 
home-based service and retention/participation in Even Start. Retention rates, defined as 
the percent of families participating in Even Start for six or more months, are substantially 
higher in projects that provide services primarily in the home than in projects that provide 
fewer home-based services (Exhibit 6.24). Program retention goes up from about 40 
percent in projects that provide no home-based services to about 70 percent in projects 
which provide 68 percent or more of their services in the home. This relationship exists 
for all three core service areas. 
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Exhibit 6.24: Percent of Even Start Families Participating for Six or More 
Months by Percent of Home-based Services and Core Service Area 




Percent home-based services 



Exhibit reads: 69 percent of children in mostly home-based projects participated in early childhood education 
for 6 or more months. 

Note: Based on reports from 1 20 of 340 projects. 



At the same time that retention is improved, service intensity is decreased in projects that 
emphasize home-based instruction. Exhibit 6.25 shows that the average hours of 
instruction per month received by participating families decreases in each core service 
area as the amount of home-based instruction increases. For example, children in projects 
which provide 33 percent or less home-based early childhood education participate for 
about 27 or 28 hours a month, while children in projects which provide more than 33 
percent home-based early childhood instruction participate for five to ten hours a month. 
This finding makes sense in that it is relatively easy to have high numbers of hours in a 
center-based early childhood education program where children participate as a group, 
while a home-based program which includes one-on-one instruction rarely will be able to 
provide more than one or two hours per week. The same holds for adult education and 
parenting education programs. 
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Exhibit 6.25: Average Hours of Services Received per Month by 
Percent of Home-Based Services and Core Service Area 




0% 1-33% 34-47% 68-100% 

Percent home-based services 



Exhibit reads: Children in projects which provide mostly home-based early childhood education received an 
average of five hours per month of early childhood education. 
Note: Based on reports from 120 of 340 projects. 
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Exhibit 7,1 

Hypothesized Sequence of Even Start Effects 



Even Start 



Process Outcomes 



Increased receipt of 
early childhood 
education 



Increased receipt 
of parenting 
education 



Increased receipt 
of adult education 



Increased receipt 
of support services 



: : is; * 



IlfJIlI 



Children 

•Improved school readiness 
'Q\ • Improved vocabulary 
• Improved emergent literacy 
skills 



Parents: Parenting and Personal 
Skills 

•Improved parenting skills 

• Improved parent/child 
interactions 

• Improved home learning 
environment 

• Decreased depression 
•Improved sense of mastery 

• Increased involvement with 
schools 



Parents: Literacy and 
Education 

•Improved functional literacy! 
•Attainment of GED 
•Increased reading/writing 



Camilioe 



Long-Term Effects 



Children 

•Improved school 
achievement 

•Improved school 
experiences (e.g., 
attendance, grade 
retention, special 
ed. placement) 




Families 

•Increased income 
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Exhibit 7.2 






Demographic Characteristics at Pretest of Families 


in Random Assignment Sites of the In-Depth Study 




Percentage of Families 


Demographic Characteristics 


Even Start 


Control 




(n=101) 


(n = 98) 


Family Configuration 






Single Parent 


40% 


39% 


Couple 


41% 


46% 


Extended family 


17% 


13% 


Other 


1% 


1% 


Education Level 






Parent has high school diploma, GED, 






or higher 


27% 


1 9% 


Spouse/partner has high school 






diploma, GED, or higher 


47% 


39% 


Employment Status 






Parent not working 


ft o o / 

83% 


82% 


Parent working part-time 


5% 


T 0/ 

7% 


Parent working full-time 


1 2% 


1 1 % 


Spouse/partner not working 


27% 


14% 


opouse/pariner worKing pan-urne 


6% 


7% 


opouse/pariner worKing Tuii-iime 


67% 


79% 


Primary Source of Income 






Government assistance 


53% 


a m n/ 

44% 


Job wages 


42% 


51% 


Alimony and child support 


2% 


1% 


Other 


3% 


4% 


Annual Income 






Less than $5,000 


52% 


44% 


5,000 - 9,999 


21% 


30% 


10,000 - 14,999 


13% 


9% 


15,000 - 19,999 


7% 


10% 


20,000 - 24,999 


3% 


6% 


25,000 or more 


4% 


1% 
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' the pretest-posttest gains among the control group from the gains among the Even Start 
group and dividing by tl\e standard deviation of the control group pretest scores. 

In addition to the In-Depth Study, the NEIS provides great amounts of pretest-posttest 
data on about 120 Even Start projects, and these data are used to provide information 
about pretest-posttest gains for Even Start participants. The approach used here is 
measure-specific. For some measures we have developed our own Even Start "norms," 
for others we use external standards of comparison such as the norms that publishers 
provide for standardized tests, the scores attained by similar populations in other recently 
completed evaluations, or the scores or program participants prior to receiving program 
services. While these approaches are second-best to a randomized experiment, the 
combination of the two provides for a strong assessment of the short-term effects of 
Even Start. 

It is important to recognize that findings from the In-Depth Study are based on a 
longitudinal study of a fixed group of families. These families were randomly assigned 
to Even Start or a control group in the fall of 1991 and were pretested. They were then 
tested twice more, at nine months and at 18 months after the pretest. All families 
participating in the evaluation were assessed at each measurement point, regardless of 
whether they were participating in Even Start. This longitudinal assessment provides an 
unbiased, and conservative, estimate of the impacts of Even Start . 

In contrast, data from the NEIS were collected under a very different design, which allows 
answers to different questions. Data for the NEIS were collected on program families 
only, at entry to Even Start, and at the end of each school year or at exit from the 
program (if the family left the program). No measures were made of any families after 
they exited from Even Start. Thus, the NEIS data are restricted to the subset of families 
who remained in the program between pretesting and posttesting. Any family that did 
not participate long enough to be posttested had to be omitted from the analysis. Sincn 
we expect length of participation to be related to positive program effects, we also expect 
families measured in the NEIS to show larger gains than program families measured in the 
In-Depth Study. Thus, findings from the NEIS can be generalized only to the population 
of families that remain in the program for several months {long enough to be posttested), 
while findings from the In-Depth Study can be generalized to all Even Start families in the 
selected projects, including those that dropped out early. 

In this report we try to reconcile the findings from these two data sets. This is easy 
when the two data sets point to the same conclusion, but it is more difficult when one 
data set points to a positive effect while the other data set shows that Even Start makes 
no difference. What should be remembered is that differences in the design of the two 
data sets mean that they are not necessarily equally appropriate for answering the same 
questions. 

A listing of the outcome measures used in the In-Depth Study portion of the evaluation 
and in the NEIS is contained in Exhibit 7.3. Outcome measures on the NEIS were 
collected only in Cohort 1 and 2 projects. 
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Exhibit 7.3 






Outcome Measures Used in In-Depth Study and NEIS 




1 

Outcome Measure | 


IDS 


NPIQ 
nicio 


PhilHrpn 






PreSchool Invantory 


X 


X 


Peabody Picture Vocabulary Test 


X 


X 


Child's Emergent Literacy Test 


X 




raisnts: Literacy levels 






Comprehensive Adult Student Assess. System 


X 


X 


MixainrTieni ot a ijcu 


X 


X 


neaoiny/wriiing aciivixies ax nome 


X 




Parents: Parenting and personal skills 






Personal skills 






Depression scale 


X 




Mastery scale 


X 




Home learning environment 






Talking with child 


X 


X 


Play materials in home 


X 


X 


Number of children's books in home 


X 


X 


Reading to child 


X 


X 


Reading materials in home 


X 


X 


Learning activities at home 


X 


X 


Teaching child 


X 


X 


iZq im ill/ n il 

ramily r UI6S 


X 




nl» U Vllloo Willi IsIlllU 


V 
A 




raffs ill aS a 1 caCnci 


X 


X 


r artsni/iuiiiiu rcaoiny idSK 


X 


-- 


raicnio cxpcoiaiiurio ctnu invuivoniorii 






CXpcSula UUilo lUF ovllUUI oUOucoo 


X 


X 


Expectations for high school grad. 


X 


X 


Families 






Social support scale 


X 




Family resources 






Source of income 


X 




Income level 


X 




Adequacy of resources 


X 




Employment status 


X 


X 
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Exhibit 8.1 






PSI Pretest Scores 






(Raw Scores from the NEIS Data Set) 




firm in II 

umup II 


N 

111 


Maon II 
IVItZCll 1 II 


SD 


Age at pretest 








O-U tO o-l 1 


963 


10.0 


5.8 


4-0 to 4-11 


1 A 1 O 

1,413 


1 A A 

14.4 


6.5 


5-0 to 5- l l 


347 


18.9 


7.0 


Gender 








ft. A I 

Male 


1,317 


13.4 


7.2 


Female 


1 ,359 


14.0 


7.2 


Ethnic Background 








Asian 


37 


1 2.4 


8.1 


African-American 


ft 

897 


13.4 


7.1 


nispanic 


TOO 

/2SJ 


J 2.0 


7.6 


Native American 


7b 


1 3.7 


6.8 


wnite 


992 


I 5.2 


6.7 


Prior preschool experience 








IMO 


1 ,94o 


1 O "7 

I 2.7 


6.9 


Yes 


697 


16.4 


7.3 


nignesi grade attained Dy target parent 








oraue u-*+ 


"7/1 

/4 


11.5 


9.0 


Uiduc o o 




1 O Q 

1 Z.9 


7.2 


Grade 9-12 


1,619 


13.8 


7.1 


Diploma or GED 


504 


14.9 


7.2 


Language of test administration 








English 


2,296 


14.3 


7.0 


Spanish 


357 


11.4 


7.7 


Total 


2,723 


13.4 


7.0 


Exhibit reads: The average PSI pretest score for males was 13.4 points. 
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Developing Age Norms for the PSI 



One way to measure the effect of Even Start on the PSI is to compare pretest-posttest 
changes observed for children in families assigned to Even Start with changes observed 
for children in families assigned to the control group in the In-Depth Study projects. 
Because families were randomly assigned to the two groups, this approach will yield an 
unbiased estimate of the effect of Even Start in these sites. But, the In-Depth Study 
relies on data from only five sites, with a total of about 100 families in Even Start and 
100 control families. 

We would like to use data from the NEIS to augment the conclusions that can be drawn 
from the In-Depth Study. The problem is that the NEIS only collects data from Even Start 
families-no control families are measured. If the PSI had national norms, we could 
estimate the amount of growth expected on the PSI by comparing the pretest-posttest 
growth of Even Start children with the growth of children nationally. But, no such norms 
exist for the PSI, and, even if they did, they probably would not be based on a sample of 
children that adequately represents the Even Start population. 

Fortunately, the pretest data collected for this evaluation afforded the opportunity to 
develop age norms for the PSI based on data collected on Ev en Start children. By 
definition, the resulting norms are directly applicable to the Even Start population. In 
brief, the methodology called for administering the PSI to children three to five years old 
as they entered Even Start and using these pretest scores to generate a growth curve 
which represents the no-treatment expectation for the Even Start population. Additional 
information on this approach is contained in St. Pierre et al., (1993) and Murray et al., 
(1993), and is reproduced in the technical appendix to this report. To the extent that 
children entering Even Start have had prior preschool experience, their pretest scores 
reflect learning obtained through that experience. 

The results of the norms development effort are summarized in Exhibit 8.2 which shows 
that children in the Even Start population are expected to gain an average of .40 items 
per month on the PSI, solely on the basis of normal development. Children who are 
administered the PSI in Spanish are expected to gain an average of .29 items per month. 
Gains are expressed in terms of number of items per month because children participate 
in Even Start for different lengths of time (different numbers of months). We use these 
no-treatment expectations in subsequent analyses to determine whether participation in 
Even Start produced pretest-posttest changes which are greater than what would be 
expected on the basis of normal development. 



Effects on the PSI 

Data from the national survey of all Even Start programs (the NEIS) indicate that Even 
Start has a statistically significant, positive effect on the PSI-an effect commensurate in 
size with effects on the PSI that have been observed in other evaluations of high-quality 
preschool programs. During the first year of the random assignment In-Depth Study, Even 
Start children gained at a significantly greater rate than control group children. However, 
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Exhibit 8.2 






Developmental Growth on the PSI in the 
Absence of Even Start Instruction 
(Based on the NEIS Data Set) 




Group 


Number of 
Children 


Expected 
Growth in 
Items/Month 


95% 
Confidence 
Band 


No prior preschool 
experience 


1,481 


.40 


.36-.44 


Prior preschool experience 


466 


.37 


.33-.42 


Administered in Spanish 


366 


.29 


.24-. 34 


Exhibit reads: Without preschool experience, including Even Start, a child would be expected to gain .40 
raw score points per month on the PSI. 


Note: Developmental growth is estimated from cross-sectional pretest scores. It is defined as the slope 
of the regression line predicting pretest scores from age in months. The 95 percent confidence band is 
obtained by adding and subtracting 1 .96 standard errors from the expected growth. 



once control group children entered Head Start or kindergarten, they gained at the same 
rate as Even Start children. 

Effects as Measured by the In-Depth Study. Data from the In-Depth Study show that 
children in families assigned to participate in Even Start gained an average of 10.2 points 
from pretest to the second posttest (about an eighteen-month period) compared to an 
average gain of 9.3 points for children in families assigned to the control group (Exhibit 
8.3). Both the Even Start and the control group gained a statistically significant amount 
on the PSI since the pretest. However, the gain for the Even Start children is not 
significantly greater than the gain for the control children. 

These results differ from these seen at the first posttest, where Even Start children gained 
significantly more than control children. Looking at the scores from Even Start and the 
control group at the three data collection points (Exhibit 8.4), we see that Even Start 
children demonstrated rapid growth between the pretest and first posttest, with a steeper 
slope than the control group. Even Start children continued on this growth trajectory 
between the first and second posttests, but control group children gained at a faster rate 
than they had between the protest and first posttest, and partially closed the gap between 
them and the Even Start group. 
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One likely explanation for these results is that children in the control group are more likely 
to be enrolled in early childhood education as they get older. Another is that children in 
Even Start at the pretest may drop out or end participation before the second posttest. 
Exhibit 8.5 shows the percentage of Even Start and control children who were enrolled 
in an early childhood education program at the pretest and both posttests. At the pretest, 
about half of the Even Start parents reported that their children were enrolled in an early 
childhood program, compared with 23 percent of the control group. The greater 
percentage among Even Start children is most likely due to the fact that the pretest 
interview could have been administered up to a month after Even Stan program 
participation began. At the first posttest, the proportion of Even Start children reported 
by parents to be in early childhood education was double that of the control group (67 
percent versus 33 percent). It was at this time point that a significant difference between 
the groups was seen in the PSI. 



Exhibit 8.5: Participation in Early Childhood Education Among 




rjc 



= 



These findings provide evidence that (1) the majority of children in the control group 
entered an early childhood program by the second posttest, and (2) about half of the 
children in the Even Start group were no longer in the program after the first posttest. 
Therefore, it is not surprising that the significant program effects on the PSI which were 
observed at the first posttest disappear at the second posttest. 

Effects as Measured by the NEIS. Data from the NEIS were analyzed differently than data 
from the In-Depth Study. Across all Even Start projects, children gained at the rate of .91 
PSI items per month, compared with a developmental expectation of .40 items per month 
for children with no prior preschool experience. Exhibit 8.6 illustrates how the PSI growth 
trajectories of children with and without Even Start diverge over time. The trajectory for 
children prior to participation in Even Start shows growth at a rate of .40 items per 
month-the rate expected on the basis of normal development. The trajectory for children 
after participating in Even Start shows growth at an accelerated rate of .91 items per 
month. 



Exhibit 8.6: PSI Growth for a Child Entering Even Start at Age Three Compared 
with Developmental Growth Estimated from Pretest Scores 




Evan Start Growth 



D«v»k>pmamal Growth 



10t> 

6; 
61 



3-6 4-0 4-6 5-0 

Age (Yeare-Months) 



Exhibit reads: A child entering Even St at age three would be expected to gain .91 items per month on the 
PSI compared with the .40 items per mu ;h growth estimated from cross-sectional data from children without 
prior preschool experience. 



The gain of .91 items per month can be expressed as the combination of .40 items per 
month due to normal development plus .51 items per month due to Even Start. We can 
therefore say that participation in Even Start more than doubles the expected rate of 
learning on the PSI. This is a substantial effect, one which is equivalent to the largest 
child-level gains observed on the PSI in other studies of preschool programs. 
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Anothe r wav to interpret the data is to express tiie size of the effect in terms of standard 
deviatljn units. Dividing the effect of .51 items per month by the PSI's standard 
deviation of 7.0 shows that participation in Even Start has an effect size of .07 standard 
deviation units per month. Findings presented earlier in this report showed that, on 
average, children participate in Even Start for about eight months. Thus, the NEIS data 
show that Even Start has an effect of .56 standard deviation units for the average Even 
Start child. This is a "medium-sized" effect by general standards of social science 
evaluations. 

Exhibit 8.7 provides a comparison of the effect observed on the PSI in the Even Start 
evaluation with effects on the PSI as seen in four other large-scale evaluations of early 
childhood education programs conducted in the 1970s and 1980s. In the other 
evaluations, the developmental gain (no-treatment expectation) on the PSI ranged from 
.4 to .5 items per month, and the gain including the effect of the program under study 
ranged from .6 to 1 .0 items per month. Children participating in Even Start fit the 
developmental pattern observed in the other studies exactly, gaining .40 items per month 
prior to entering Even Start. Once in Even Start, they gained at an improved rate of .91 
items per month. This accelerated rate of learning on the PSI means that as Even Start 
children enter the public schools they are more likely to know basic concepts and 
precursors of kindergarten skills than they would have been in the absence of the 
program. 

Information on how different subsets of Even Start children performed on the PSI is 
presented in Exhibit 8.8. The average monthly gains are remarkably stable across 
subgroups. That is, childrens' monthly gains on the PSI are roughly equivalent, regardless 
of age of child, gender, ethnic background, prior preschool experience, education level of 
parent, language in which the test was administered, family structure, or annual family 
income. This argues that Even Start does equally well at teaching school readiness skills 
to quite varied groups of children, and that the overall estimate of .91 ; *ems per month 
is a robust indicator. Where it appears that large differences exist between subgroups, 
the sample size in one of the subgroups tends to be small, casting doubt on the reliability 
of the estimate. 



Child Literacy as Measured by The Peabody Picture 
Vocabulary Test 

The Peabody Picture Vocabulary Test-Revised (PPVT) measures receptive (hearing) 
vocabulary, and gives a quick estimate of verbal or literacy-related skills. 



Description of the Measure 

The PPVT is an individually administered test that requires 1 5 to 2.0 minutes per child and 
is appropriate for children between the ages of two and 1 8 years. In this evaluation, the 
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Exhibit 8.8 

Average Monthly Gain in Raw Score 
Points on the PSI for Selected Variables 
(NEIS Data Set) 


Group 


j Monthly Gain 


I N 
* 


Mean 


SD 


Age at pretest 








3-0 to 3-1 1 


1,127 


.85 


.88 


4-0 to 4-1 1 


1,529 


.96 


.81 


5-0 to 5-1 1 


74 


.87 


.96 


Gender 








Male 


1,288 


.89 


.86 


Female 


1,316 


.94 


.83 


Ethnic background 








Asian 


46 


1.16 


.77 


African-American 


831 


.92 


.86 


Hispanic 


681 


.95 


.96 


Native American 


85 


.87 


.82 


White 


1,003 


.88 


.75 


Prior preschool experience 








No 


988 


.99 


.86 


Yes (prior to Even Start) 


275 


.98 


.76 


Yes (Even Start prior to age 3) 


1,378 


.84 


.83 


Highest grade attained by target parent 








Grade 0-4 


91 


.90 


.99 




492 


.91 


.94 


Grade 9-1 2 


1,520 


.91 


.81 


Diploma or GED 


488 


.95 


.76 


Language of test administration 








English 


2,208 


.91 


.82 


Spanish 


430 


.92 


1.00 


Both 


75 


.90 


.82 
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Exhibit 8.8 
(continued) 








Average Monthly Gain in Raw Score 
Points on the PSI for Selected Variables 
(NEIS Data Set) 










Monthly Gain 


Group 






N || 


Mean 


SD 


Family structure 












Single parent 






932 


.93 


.89 


txtenoea iamiiy 






343 


.93 


.90 


wOupic 






1,385 


.90 


.79 


Annual income 












Under $5,000 






925 


.95 


.91 


$5,000 - $9,999 






732 


.86 


.84 


$10,000 - $14,999 






485 


.89 


.79 


$15,000 - $19,999 






245 


.93 


.74 


$20,000 - $24,999 






130 


.99 


.88 


$25,000 or more 






93 


1.02 


.76 


Total 






2,730 


.91 


.85 


Exhibit reads: Children tested twice with the PSI gained an average of .91 raw score points per month. Since 
developmental growth accounts for about .40 raw score points per month, Even Start early childhood education 
more than doubles the growth rate of readiness skills for participating children. 



the delayed test-retest reliability is .78. A great deal of validity information is available 
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Exhibit 8.9: Average PPVT Pretest 
Standard Scores by Age in Months 



-N. 



« v 



50 



0 i i i i — r~i — i — r~r 
36 42 



~i i i i i i i n i i i i i i i i i i i i i i i i i i i i i i i i i~ 
48 54 60 66 72 78 



~i — i — i — i 
84 



program effect (Exhibit 8.10). A graph of scores over the three data collection points 
confirms that there are few differences between the program and control groups (Exhibit 
8.1 1). Earlier in this discussion we saw that most children in the In-Depth Study control 
group had entered an early childhood education program by the time of the second 
posttest. This could partly explain the pattern seen in Exhibit 8.1 1, where Even Start 
appears to be making a difference at the first posttest (though not a statistically 
significant one), but the control group has moved ahead a bit by the second posttest 
(again, not significantly so). 

Effects as Measured by the NEIS. Data from the NEIS lead to a different conclusion than 
that which was drawn from the In-Depth Study data. Based on the NEIS, we see that 
Even Start children make significant gains on the PPVT. Exhibits 8.12 and 8.13 
present the results of analyses based on NEIS data from all Even Start projects. The 
exhibits show average pretest-posttest gains on the PPVT and TVIP, expressed in 
standard score points per month (gains are expressed in points per month because 
children participate for different numbers of months). 

Based on the norms analyses described above, we do not expect to see any change in 
PPVT standard scores due to maturation. However, we do see that children participating 
in Even Start gain an average of .94 standard score points a month on the PPVT and .92 
points per month on the TVIP. Dividing by the PPVT standard deviation yields a per 
month effect of .06 standard deviation units. Multiplying by eight months, the average 
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Exhibit 8.10 








Peabody Picture Vocabulary Test: 
Effects from the In-Depth Study 










Even Start 
(n = 76) 


Control 
(n = 70) 






Mean || S.D. 


Mean | 


S.D. 


Pretest 




76.6 19.5 


76.9 


17.2 


Second Posttest 




83.7 17.1 


85.1 


17.8 


Gain 




7.1* 


8.2* 




Program Effect 
(Effect Size) 




-1.1 points 
(.06 s.d.) 




• p < .05 










Exhibit reads: Even Start children in the In-Depth Study averaged 76.6 points on the PPVT pretest. 



Exhibit 8.1 1 : Peabody Picture Vocabulary Test: Pretest and 
Posttest Scores from the In-Deptn Study 

90 n 




70 -I , , 

PratMt <Ae» 4 yr».. 0 mot.) Posttut #1 (Ag« 4yn»., 9 mot,) Po«ttMt #2 (AoN 5 yr».. 6 mo».) 



Note: Based on sample of children with data at all three testing points (Even Start n = 76; control n = 70). 
Exhibit reads: Even Start children received an average standard score of 83.7 at the second posttest. 
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Exhibit 8.12 

Average Monthly Gain on the PPVT 
in Standard Score Points for Selected Variables 
(NEIS Data Set) 


Group 


Monthly Gain 


N 


Mean 


SD 


Age at pretest 








3-0 to 3-1 1 


858 


.39 


2.1 1 


4-0 to 4-1 1 


1,309 


1.47 


2.30 


5-0 to 5-11 


571 


.89 


1.92 


6-0 to 6-1 1 


289 


.51 


1.86 


7-0 to 7-11 


92 


.34 


2.19 


Gender 








Male 


1,528 


.99 


2.20 


Female 


1,461 


.86 


2.15 


Ethnic background 








Asian 


50 


1.52 


2.62 


African-American 


1,091 


1.14 


2.35 


Hispanic 


382 


.81 


2.37 


Native American 


177 


.22 


1.61 


White 


1,324 


.89 


1.96 


Prior preschool experience 








No 


961 


1.21 


' 2,1 9 


Yes (prior to Even Start) 


504 


1.03 


2.14 


Yes (Even Start, prior to age 3) 


1,545 


.76 


2.19 


Highest grade attained by target parent 








Grade 0-4 


33 


.65 


3.17 


Grade 5-8 


460 


.97 


2.17 


Grade 9-12 


1,843 


.96 


2.19 


Diploma or GED 


626 


.87 


2.10 


Form administered at pretest 








Form L 


2,904 


1.00 


2.18 


Form M 


215 


.10 


2.14 j 
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Exhibit 8.12 
(continued) 

Average Monthly Gain on the PPVT 
in Standard Score Points for Selected Variables 
(NEIS Data Set) 



Group 



Monthly Gain 



N 



Mean 



Family structure 

Single parent 
Extended family 
Couple 
Annual income 



1,214 
349 
1,476 



1.02 
1.15 
.84 



SD 



2.30 
2.13 
2.12 



Exhibit 8.13 








Average Monthly Gain on the TVIP 
in Standard Score Points by Age at Pretest 






Monthly Gain 


Age at Pretest 


N 


Mean 


SD 


3-0 to 3-1 1 


148 


.48 


2.45 


4-0 to 4-1 1 


202 


1.08 


2.45 


5-0 to 5-1 1 


129 


1.30 


2.26 


6-0 to 6-11 


78 


.56 


2.12 


7-0 to 7-1 1 


24 


1.31 


1.84 


Total 


581 


.92 


2.36 


Exhibit reads: Children tested twice with the TVIP had an average monthly gain of .93 in standard score 
points. 


Note: Combined norms (Mexico and Puerto Rico) were used in obtaining standard scores. 





Child's Emergent Literacy Test 



In addition to the PSI and PPVT, we assessed children in the In-Depth Study using a brief 
set of items desianed to tao children's emeraent literacy skills. This measure was not 



couple of minutes to administer. The testing was done at the same timepoints as the PSI 
and PPVT. 



Because the CELT was created for this evaluation, there is no history of psychometric 
characteristics. The reliability (Cronbach's alpha) of the CELT as administered in this 
evaluation is .76. The CELT consists of 16 items, each is scored correct or incorrect, 
with a total score ranging from zero to 16. 



Pretest Levels on the CELT 

Children in Even Start and in the control group scored quite similarly on the CELT at 
pretest: the Even Start pretest mean was 4.1 and the control group pretest mean was 
4.5 (see Exhibit 8.14). Standard deviations were almost identical: 2.9 for Even Start and 
3.1 for the control group. We have no basis of comparison for these pretest scores since 
the instrument has not been used in other studies. We should note that ceiling effects 
are not a problem, since pretest means are between four and five points out of a total of 
16. However, the low pretest scores may indicate that the measure is too difficult, and 
hfinne is not sensitive to aaips for children at this age. 
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child's posttest PPVT score than demographic variables such as 
family income or the number of children's books in the home. 

These analyses were based on 614 families with data for all of the variables in the 
regression. The final regression model predicted the PPVT posttest, controlling for the 
PPVT pretest, CASAS pretest, number of hours in early childhood education, and number 
of hours in parenting education. Together, these variables explained 42 percent of the 
variance in the PPVT posttest scores. 

Exhibit 8.15 shows how gains on the PPVT are related to the amount that Even Start 
children participate in early childhood education and to the amount that their parents 
participate in parenting education: 

• Even Start children are expected to gain 1.1 points on the PPVT for each 
100 hours that they participate in an early childhood education program. 



Even Start children are expected to gain an additional 1 .4 points on the 
PPVT for each 100 hours that a parent participates in parenting education. 







Exhibit 8.15: 


PPVT Gains as Predicted by Amount of 


10-; 


Early Childhood Education and Amount of Parenting Education 
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Exhibit reads: Even Start children gain 1 . 1 points on the PPVT for each 1 00 hours that they participate in Early 
Childhood Education. They gain an additional 1.4 points on the PPVT for each 100 hours that a parent 
participates in Parenting Education. 
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We can translate these findings into information that is useful to program planners by 
showing the PPVT scores (and their percentile equivalents) that are expected to result 
from some typical combinations of early childhood education and parenting education 
programs (Exhibit 8.16). The exhibit shows three levels of intensity of parenting 
education: 36 hours-corresponding to a 1-hour per week program lasting for a 36-wsek 
school year; 108 hours-corresponding to a 3-hour per week program lasting for a 36- 
week school year; and 180 hours-corresponding to a 5-hour per week program lasting 
for a 36-week school year. It also shows two levels of intensity of early childhood 
education that could be included as part of a typical preschool program: 270 hours- 
corresponding to a 3-day, 2.5 hour per day program lasting for a 36-week school year; 
and 540 hours-corresponding to a 5-day, 3 hour per day program lasting for a 36 week 
school year. 



Exhibit 8.16 

PPVT Standard Scores for the Average Even Start Child as Predicted* 
by Intensity of Early Childhood Education and Intensity of Parenting Education 



Intensity of 

Parenting 

Education 



36 hours 
(1 hour/week * 36 
week school year) 



108 hours 
(3 hours/week * 
36 week school 
year) 



180 hours 
(5 hours/week * 
36 week school 
year) 



Intensity of Early 
Childhood Education 



270 hours 
(2.5 hours/day * 3 
days/ week * 36 
week school year) 



85.6 points 
(1 7th percentile) 



86.6 points 
(19th percentile) 



87.6 points 
(20th percentile) 



540 hours 
(3 hours/day * 5 
days week * 36 
week school year) 



88.6 points 
(22nd percentile) 



89.6 points 
(24th percentile) 



90.6 points 
(26th percentile) 
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Even Start children participating in a relatively low-intensitv program consisting of 1 hour 
per week of parenting education combined with 7.5 hours per week of early childhood 
education would be expected to have a PPVT posttest score of 85.6 points, 
corresponding to the 17th percentile nationally. On the other hand, Even Start children 
participating in a high-intensity program consisting of 5 hours per week of parenting 
education combined with 15 hours per week of early childhood education would be 
expected to have a PPVT posttest score of 90.6 points, corresponding to the 26th 
percentile nationally. 

Two reference points help in interpreting the expected scores shown in the exhibit: 

• Average ECE and PE: 8 The expected PPVT posttest score for an Even Start 
child who received the average amount of early childhood education (268 
hours) and whose parents received the average amount of parenting 
education (100 hours) is 86.5 points (19th percentile). 

• No ECE or PE: An Even Start child who receives no early childhood 
education and whose parents receive no parenting education is expected to 
have a PPVT score of 82.1 points (12th percentile). When placed in a high- 
intensity Even Start program for a year, we expect that same child to have 
a PPVT score of 90.6 points (26th percentile). 

It is important to realize that the levels of programmatic intensity described above are 
provided for purposes of illustration only-so thai the reader can see how some typical 
programmatic approaches to an Even Start program would be expected to affect PPVT 
scores. However, we do not know whether there is any special configuration of services 
that works better than other configurations. For example, we do not know whether 500 
hours of early childhood education received in one year is more or less effective than 500 
hours accumulated over two years. 
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necessarily be expected to relate to the PPVT. Parenting education activities often are 
targeted at developing parent's abilities as teachers of their children, and children's 
language development is exactly what is taught through many of the parenting activities 
emphasized in Even Start, such as reading to children. On the other hand, there is no 
special reason that participation in adult education programs, which focus only on the 
development of adult-level skills, should yield immediate benefits for children. 

Finally, a similar analysis was conducted to determine whether amount of parenting 
education was related to PSI posttest scores. While we noted some of the same 
relationships seen in the PPVT analysis (e.g., a positive relationship between the amount 
of time children spend in early childhood education and PSI posttest scores), the amount 
of time spent in parenting education was not significantly related to PSI posttest scores. 

It is likely that parenting education contributes to improved PPVT scores but not PSI 
scores precisely because parenting education often focuses on reading to children, which 
in turn enhances language development. The skills measured by the PSI are more readily 
learned in a preschool setting than those measured by the PPVT. 



Effect of Service Variations on Child Outcomes 

Service Intensity. One hypothesis about Even Start is that higher intensity services ought 
to lead to greater gains on outcome measures. Our study design does not allow us to 
address this question directly, because we have not randomly assigned families to 
projects offering different levels of service intensity. Still, data from the NEIS offer the 
opportunity to examine the relationship between service intensity and outcomes as it 
occurs naturally. The drawback to this approach is that we cannot disentangle the 
observed relationship from characteristics of families (e.g., level of need, motivation) that 
predispose them to participate more or less fully in Even Start. 

With this caveat in mind, data from the NEIS show a statistically significant, positive 
relationship between total hours of early childhood education and total gains on the PSI 
(Exhibit 8.17). Each 100 hours of exposure to early childhood services translates into a 
gain of about 1.2 points on the PSI (about 1/6 of a standard deviation). Also, these 
findings for the PSI are consistent with findings on the relationship between amount of 
service and PPVT scores, presented in the preceding section. Though not shown 
graphically, a similar relationship exists between monthly hours of early childhood 
education and monthly gains on the PSI and PPVT. Both of these findings make sense: 
greater exposure to an early childhood education setting results in larger gains. 

Home-Based Services. Another question that has been raised is whether the extent to 
which services are home-based has an effect on outcomes. We have the same caveat 
for this analysis as for the analysis of the effect of service intensity on cKJ outcomes- 
families have not been randomly assigned to projects that offer different levels of home- 
based services. In any case, data from the NEIS show no strong relationship between 
extent of home-based services and either PSI or PPVT scores. 
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Exhibit 8.1 7: PSI Gains by Total Hours 
of Early Childhood Education (NEIS Data Set) 
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Exhibit reads: Children who received a total of 1 to 120 hours of early childhood education gained an average 
of 5.4 items on the PSi. 



Quality of Staffing. A final question that has been posed is whether the quality of the 
staff in an Even Start project is related to child outcomes. This analysis is subject to the 
same caveats as the above analyses. Further, our attempts to create project-level staffing 
indicators from data on years of experience, formal education of instructional staff, and 
inservice training were not very successful, and there were quite low correlations between 
these indicators and other project characteristics and outcomes. 

A major limitation of the staffing data is that background information on staff from 
collaborating agencies was not collected, and it is exactly these staff members who often 
are responsible for the direct provision of core services. Even when analyses were limited 
to projects in which Even Start staff were the primary providers of adult education and 
early childhood education, we observed low correlations between staff characteristics and 
outcomes. We suspect that the data collected on staffing may not be adequate to 
support the type of analyses we hoped to perform. 



Conclusions About Effects of Even Start on Children 

This evaluation assessed effects on children using three different measures: (1) the Pre- 
school Inventory, a measure of school readiness; (2) the Peabody Picture Vocabulary 
Test, a measure of hearing vocabulary; and (3) the Child's Emergent Literacy Test, a 
measure of emergent literacy skills. 
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On each of the measures, there are no statistically significant differences between 
children in Even Start and the control group at the second posttest using data from the 
In-Depth Study. While these results are discouraging, they need to be considered in light 
of two findings that work against program effects: (1) the majority of children in the In- 
Depth Study control group were enrolled in some type of early childhood program by the 
second posttest, and (2) nearly half of the Even Start children in the In-Depth Study were 
no longer enrolled in an Even Start early childhood program by the second posttest. The 
impact of this involvement by the control group in early childhood education is seen most 
dramatically on the PSI, where significant program effects reported at the first posttest 
erode by the second posttest. The results suggest that, when control group children 
enroll in an early childhood program, they learn many of the same skills the Even Start 
children learned in preschool. What is not known is the long-term effects of this lag time 
in learning. Since most of the control group children are in kindergarten by the second 
posttest, it appears that they are learning many of the basic readiness skills (e.g., shapes, 
colors, directionality) during their first year of public school. If many of the Even Start 
children learned these skills in a preschool program, they may be progressing to other 
school skills (e.g., prereading) during their kindergarten year. Thus, the finding that the 
differences between Even Start and control children on basic readiness skills are 
diminished when control children reach kindergarten does not preclude other longer-term 
impacts of Even Start on school performance. 

In contrast to the In-Depth Study, data from the NEIS show positive gains on the PSI and 
PPVT, above and beyond what would be expected in the absence of the program. 

A key difference between the NEIS and In-Depth Study is the inclusion or exclusion of 
families no longer in Even Start. As described in the general approach to the analysis, 
children were posttested for the NEIS at the end of the program year or just prior to 
program exit. In contrast, children in the In-Depth Study were tested at fixed time points 
regardless of whether they were still in Even Start. Thus, the NEIS can address the 
question of the potential impact of Even Start for families who remain in the program, 
while the In-Depth Study offers a longitudinal look at children who had varying levels of 
participation in the program. 

Attributing the gains seen on the NEIS to Even Start is given some credibility by analyses 
of the relationship between program participation and outcomes within the Even Start 
group. Greater gains are seen on the PSI and PPVT both for children (a) with more 
hours in an Even Start early childhood education program and (b) whose parents logged 
more hours in parenting education. These results suggest encouraging program effects. 
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This evaluation measured the effects of Even Start on the literacy skills of parents in the 
following areas: (1 ) functional literacy level on a reading test, (2) the percentage of Even 
Start adults who obtained their GED certificate, and (3) parental reports of their own 
reading and writing activities in the home. This section of the report includes discussions 
of: 

• Functional Literacy of Adults as Measured by the CASAS 

• Attainment of a GED Certificate 

• Reading and Writing Activities in the Home 

• Conclusions about Effects of Even Start on Parent Literacy Skills 



Functional Literacy of Adults as Measured by The 
Comprehensive Adult Student Assessment System 

The Comprehensive Adult Student Assessment System (CASAS) is an adult-oriented 
functional assessment system that measures a broad range of adult literacy skills and 
their application in real life domains including consumer economics, government and law, 
occupational knowledge, community resources, and health (Rickard et al., 1990). 
Although the CASAS measures reading, writing, math and problem solving skills, this 
evaluation used only the Reading Survey achievement test in order to reduce respondent 
burden and because we expected Even Start's effects to be more prominent in reading 
than in math. 



Description of the Measure 

The CASAS has the flexibility to measure participants involved in diverse adult education 
programs, spanning the range from non-readers to adults at the GED or high school level. 
An untimed paver-and-pencil test, the CASAS Reading Survey may take as long as 60 
minutes to complete. The CASAS has been used with adult education learners in 27 
states. The test is used in adult education and in job training programs, with both native 
and non-native English speakers. It also has been accepted as a project in the National 
Diffusion Network. Thp HAS AS is hninrL jirrH in tho Nat ional Fwaliiatinn nf AHutt 



GAIN program (CASAS, 1990), and in the evaluation of California's 321 adult education 
programs (CASAS, 1991). 

CASAS reading scores range from 1 50 to 260. The test developers suggest the following 
interpretation of CASAS scale scores: 

• Beginning literacy (below 200): Adults scoring below a scale score 
of 200 have difficulty with the basic literacy skills needed to function 
in an employment setting and in the community. While these adults 
can handle routine, entry-level jobs, they may have trouble following 
simple directions and safety procedures. 

o Basic literacy (200 through 214): Adults scoring between scale 
scores 200 and 214 can function in entry-level jobs that require only 
minimal literacy skills. They can fill out simple applications. 

• Intermediate literacy (215 through 224): Adults scoring between 
scale scores 215 and 224 are able to perform basic literacy tasks in 
a functional employment setting. They are generally able to function 
in jobs or job training that involves following written instructions and 
diagrams, though they usually have trouble following complex sets 
of directions. 

• High school literacy (225 and above): Adults scoring above a scale 
score of 224 can usually perfoim work that involves written 
directions in familiar and some unfamiliar situations. They can 
function at a high school entry level in basic reading and, if they do 
not have a high school diploma, can profit from instruction in General 
Education Development and have a high probability of passing the 
GED test in a short time. 

Sticht (1990) found these interpretations to be reasonable and reported general 
correspondence between CASAS scale scores above 225 and the ninth to twelfth grade 
reading levels on the Tests of Adult Basic Education (TABE) and the Adult Basic Learning 
Examination (ABLE). We have found minimal data on the psychometric characteristics 
of the CASAS. A correlation of .70 between the CASAS reading test and the ABLE was 
reported in unpublished data. To obtain an estimate of test-retest reliability, we 
calculated the correlation between pretest and posttest scores for adults who were 
posttested less than 90 days after the pretest. The correlation was .86, suggesting that 
the CASAS is a reliable measure. The true test-retest reliability might be even higher 
since this estimate is based on data using alternate forms of CASAS tests. 

For the NEIS, Even Start staff administered the CASAS Reading Survey. The test has four 
levels, A through D, with 24 to 40 items per level and alternate forms of each level. 
Scale scores link the levels into a continuous scale of achievement. Staff administered 
a short "locator" test to assist in identifying the appropriate level of the CASAS. There 
is no Spanish version of the CASAS, and project staff were instructed to use their own 
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judgment as to the appropriateness of administering the test to adults with limited reading 
skills in English. For the In-Depth Study, the CASAS was administered by trained data 
collectors using the same schedule and rules as all other In-Depth Study measures. 



Effects on the CASAS 

Data to assess program effects on the CASAS are available from the In-Depth Study and 
from the NEIS. The NEIS data show pre-post gains that are statistically significant and are 
about Va standard deviation in size. These gains are as large or larger than those seen in 
other evaluations of adult education programs. The In-Depth Study data show statistically 
significant gains on the CASAS for both the Even Start and control groups, but there is 
no significant effect of Even Start on this measure. Both data sets show a significant 
relationship between amount of adult education instruction and gains on the CASAS. 

Effects as Measured by the In-Depth Study. Adults in the In-Depth Study were pretested 
and then posttested twice: once nine months after the pretest, and again 18 months 
after the pretest. Comparing scores at the pretest to scores at the second posttest we 
see that adults in Even Start gained an average of 3.7 points on the CASAS, compared 
to 3.6 points for adults in the control group (Exhibit 9.1). Both of these gains are 
statistically significant. However, the gains for the Even Start adults are not appreciably 
larger than those of the control group. Graphing the CASAS scores at the three 
measurement points shows that gains in the Even Start and control groups are 
comparable across time, although the Even Start group consistently scores at a higher 
level than the control group (Exhibit 9.2). 

In an effort to understand more about this pattern of gains, we examined data on the 
extent to which adults in Even Start and in the control group participated in adult 
education activities. At each data collection point, adults in Even Start and the control 
group were asked whether they had participated in adult education classes during the 
previous year (adults in the control group were free to participate in any educational 
programs offered in the community). 

Exhibit 9.3 shows that a larger percentage of Even Start adults than control group adults 
(47 percent vs. 26 percent) reported that they participated in adult education prior to the 
pretest. 9 This was unexpected, because the groups were randomly assigned and were 



9 These participation rates are based on parents' self-report and may underestimate 
actual participation. However, a comparison of self-report data for the Even Start group 
with contact log data supplied by Even Start project staff shows fairly high overlap 
between the two data collection methods. For example, at the first posttest, only 1 1 
percent of adults in Even Start indicated that they did not participate in adult education 
when the program's contact log data recorded more than 20 hours of adult education. 
It is possible that project staff who completed the contact logs used a broader definition 
of adult education (e.g., individual tutors) than the parents themselves used. These 
results suggest that the self -report data are valid, and we use them in these analyses in 
order to have comparable data sources for the program and control groups. 
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Exhibit 9.1 






CAS AS Reading Survey: 
Effects from the In-Depth Study 
(Scale Score Points) 
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Control 
(n = 53) 




Mean f S.D. 
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S.D. 


Pretest 


229.4 13.7 


226.8 


16.9 


Second Posttest 


233.1 13.0 


230.4 


14.1 


Gain 


3.7* 


3.6* 




Program Effect 
(Effect Size) 


0.1 points 
(.01 s.d.) 




• p < .05 








Exhibit reads: Even Start adults in the In-Depth Study averaged 229.4 points on the CASAS Reeding 
Survey pretest. 



Exhibit 9.2: CASAS Reading Survey: 
Pretest and Posttest Scores from the In-Depth Study 

235 n 




220 i , . 

Pretest First Posttest Second Posttest 



Exhibit reads: Even Start adults scored 233.1 on the CASAS at the second posttest. 

Note: Only adults with CASAS scores at the three data collection points are represented on this graph (Even 
Start n = 64; control n = 53). 
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Exhibit 9.3: Participation in Adult Education: 
Results from the In-Depth Study 
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Exhibit reads: At the second posttest, 63.3 percent of Even Start aduhs reported participating in adult education 
during the past year. 

Note: Data are based on adult's self-reports of participation in ABE, ESL or GED classes for adults with 
information at three points (Even Start rt = 79; control n = 69). In some cases pretesting was done shortly 
after program participation began. This accounts for the fact that Even Start adults participate in adult 
education at a greater rate than control group adults at the pretest. 

comparable on many other background characteristics. It may be an indication that, for 
some adults, the pretest actually was given after Even Start adult education instruction 
had begun. Further, it is a possible explanation for the fact that Even Start adults scored 
higher than control group adults at the pretest. 

As expected, Even Start adults reported significantly higher participation rates in adult 
education programs at both posttests than did control group adults. Still, approximately 
one-quarter of adults in the control group indicated that they were involved in either adult 
basic education (ABE) classes, preparation for the GED certificate, or ESL classes at the 
time of the pretest and both posttests. 

Looking at the type of adult education in which adults participated, we see that adults in 
Even Start are more likely than adults in the control group to be taking ABE or GED 
classes (Exhibit 9.4). Among Even Start adults, 41 percent reported participating in a 
GED class in the year prior to the second posttest, compared with 15 percent of the 
control group, and 27 percent of Even Start adults are involved in adult basic education 
classes, compared with 9 percent of the control group. Both of these differences are 
statistically significant. However, the proportion of Even Start adults taking ESL classes 
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Exhibit 9.4: Type of Adult Education at Second Posttest: 
Results from the In-Depth Study 
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Exhibits reads: In the In-Depth Study projects, 27.1 percent of Even Start adults reported participating in adult 
basic education in the year prior to the second posttest. 



(27 percent) is not significantly higher than the proportion of the control group (15 
percent). 

The CASAS results reported here show positive changes over time for adults in Even Start 
and the control group. About one-quarter of adults in the control group reported that they 
participated in adult education programs, although this does not appear to explain their 
gains on the CASAS, 10 These results suggest that the CASAS test might not be 
sensitive to certain types of instruction, such as preparation for the GED certificate. In 
addition, it appears that gains on the CASAS test can be expected in the absence of 
direct instruction. These gains also might be due to a testing effect, where adults are 
more relaxed and experienced about the requirements of the test at the posttest. 

Effects as Measured by the NEIS. Analyses of data from the NEIS were limited to adults 
who had a valid pretest and posttest, who had at least three months between pretest and 
posttest, and who participated in at least 70 hours of adult education instruction between 
pretest and posttest. These criteria ensured that there was adequate time between test 
points for gains to occur, and that there was sufficient instruction to produce gains (the 



10 CASAS gains for adults who reported that they participated in adult education were 
no different than gains for adults who did not participate. 
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CASAS developers do not expect to see educationally significant gains with less than 70 
to 100 hours of instruction). 

If Even Start has had a positive effect on adult literacy, there should be improvement on 
the CASAS scale scores between pretest and posttest. We do not have a control group 
for the NEIS data, nor do we have a theory (as we do for the PSI or PPVT) that a portion 
of observed pre-post changes are due to some developmental process (although data from 
the In-Depth Study show that the control group gains over time). 

Analysis of data from the NEIS shows that adults participating in Even Start for 70 hours 
or more over at least a three-month period gain an average of 4.6 scale score points on 
the CASAS. This gain is statistically significant, not a surprising result given the large 
number of adults included in the analysis. The effect size for the CASAS (gain divided 
by the CASAS standard deviation) is .31 standard deviation units, a value that is generally 
considered small in educational research (Exhibit 9.5). 

CASAS Gains in Even Start vs. Other Adult Education Programs. Given this real but 
relatively small gain, and the fact that the In-Depth Study does not show any program 
effect, we asked whether this finding is unique to Even Start or if other adult education 
programs also have difficulty producing measurable gains. There have been few large- 
scale evaluations of adult education programs that contain outcome data that may be 
used for comparison purposes. Exhibit 9.6 compares the effect size for the CASAS gains 
of Even Start adults with the results from studies of other programs. 

In one evaluation of federal 321 -funded adult education programs in California, 
researchers (CASAS, 1992) found average gains of 3.0 scale score points (effect size: 
.20) for 973 students participating in adult basic education and 3.3 points (effect size: 
.22) for 1,757 students participating in English as a second language programs. All 
students took the CASAS Reading Survey after 80 to 1 00 hours of instruction. An earlier 
study of statf-funded competency-based adult education programs in California (CBAE, 
1987) found gains of 4.2 scale score points (effect size: .29) after 100 hours of 
instruction, although the measure was the CASAS Survey Achievement battery which 
includes fewer items to measure a wider range of literacy skills than the Reading Survey 
test. In an evaluation of the Kenan model, which is used by many Even Start projects, 
researchers (Darling and Hayes, 1989) found small gains on the reading subtest of the 
TABE (effect size: .29) over one project year covering an unspecified number of 
instructional hours. 

These results show that Even Start is of comparable effectiveness o other adult 
education programs in improving the literacy skills of participating adults although the 
literacy gains in all of these programs appear to be moderate to small. This is reasonable 
since many Even Start projects use services provided by local adult education programs 
for this core service. 

CASAS Gains by Hours of Instruction. The wide range in the length of time that families 
participate in Even Start and in the intensity of the adult education instruction provided 
by projects furnishes us an opportunity to strengthen our conclusions about CASAS gains. 
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CAS AS Reading Survey: 
Gains from the NEIS 
(Scale Score Points) 
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Exhibit reads: Adults participating in 70 or more hours of adult education gained 4.6 scale score 
points on the CASAS reading. 
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Exhibit 9.6: Comparison of Effect Sizes for Even Start (NEIS Data Set) 
and Three Adult Education Programs 
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Exhibit reads: In Even Start, the effect size for gains on the CASAS is .31 standard deviation units. 



If adults with many hours of adult education instruction improve more than adults who 
participate only for a few hours, we can be more confident that the instruction (and 
hence. Even Start) is responsible for this growth. It also is important to examine the 
effect of entry level on CASAS gains. Due to "ceiling effects" we expect smaller gains 
for adults starting near the highest valid CASAS score than for adults entering with lower 
entry scores. 

Exhibit 9.7 shows CASAS reading gains in the NEIS for subgroups of adults defined by 
the number of hours of adult education. Adults who participated for less than 70 hours 
gained about 3.0 CASAS scale score points, adults who participated for 70-200 hours 
had an average gain of 4.3 points, and adults who participated for over 200 hours gained 
an average of 5.2 points. This relationship is statistically significant and provides some 
evidence that the adult education instruction provided through Even Start may be 
responsible for the gains reported earlier. 

Among the program families in the In-Depth Study, there also is a greater gain on the 
CASAS for adults with more than 200 hours of adult education. These adults gained an 
average of 6.8 points from pretest to second posttest, compared to gains of 3.4 points 
for adults with less than 200 hours. The incremental gain in CASAS scores for each levei 
of adult education is not seen in the In-Depth Study because many families in the middle 
level of adult education hours dropped out of the program between the first and second 
posttests. 
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Exhibit 9.7 






CASAS Reading Gains by Hours of Instruction 
(Scale Score Points; NEIS Data Set) 
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Over 200 
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Exhibit reads: An average gain of 4.3 points on the CASAS was observed for adults with more than 
70 hours of adult education. Even higher gains were observed with more than 200 hours. 



The CASAS pretest and hours of adult education instruction accounted for about 88 
percent of the variation in CASAS posttest scores on the NEIS. Exhibit 9.8 illustrates the 
relationship by estimating CASAS gains for selected entry level scale scores and hours 
of instruction. As can be seen, the greatest gains can be made with adults who enter 
with low CASAS scores. In fact, the exhibit shows that changes in posttest scores are 
much more sensitive to the starting point of the adult than they are to the number of 
hours of adult education instruction. 



Attainment of A General Education Development Certificate 

Even Start projects promote attainment of a high school diploma or a high school 
equivalency diploma such as the GED, as do most adult education programs. The 
rationale for this goal is based upon a belief that a diploma or GED increases the chance 
the adult will find employment, provides opportunities for higher wages, and improves the 
quality of life through enhanced self-esteem. 



Description of the Measure 

For the NEIS, projects reported each June whether Even Start adults attained a diploma 
or GED during the reporting year. We calculated the percentage of Even Start adults who 
attained a diploma or GED during the year. Since adults attending a secondary school do 
not qualify for Even Start, we can assume that the reported data generally reflected 
at tainment of a GED, although other alternative diplomas are possible in some states. For 
the In-Depth Study, we calculated the percentage of adults in Even Start and in the 
control group who received their GED in the time butween entry to the study and the time 
the second posttest was administered, eighteen months later. 
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Exhibit 9.8 

Expected CASAS Reading Gains 
by Entry Level and Hours of Instruction 
(Scale Score Points; NEIS Data Set) 
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3.1 


3.3 


3.5 
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3.9 


4.1 


240 


0.5 


0.7 


0.9 


1 .1 


1.3 


1.5 



Note: These expected gains are in scale score points. The estimates are based upon a regression 
equation (R = .88) of: posttest - ( 63.389 + .737 x pretest + .004 x hours ) 

Exhibit reads: Adults who entered Even Start with a CASAS score of 200 would be expected to gain 
11.2 points after 100 hours of adult education instruction. 



Effects on GED Attainment 

Findings from both the In-Depth Study and the NEIS show that Even Start has hai a 
statistically significant, positive effect on GED attainment. 

Effects as Measured by the In-Depth Study. Data from the In-Depth Study show that 
22.4 percent of participating adults in Even Start families attained a GED during the 
course of this study (an 1 8-month period) compared to 5.7 percent of participating adults 
in control group families (Exhibit 9.9). This is a statistically significant effect that also is 
large enough to be educationally meaningful. 

We also examined, at each of the three data collection points, the percentage of adults 
in Even Start and the control group who had a high school diploma or GED (Exhibit 9.10). 
A smaller percentage of adults in the Even Start group had their high school diploma or 
GED at the time of pretest than was true of the control group (21 percent vs. 29 percent). 
This difference is not statistically significant, and is most likely an artifact of random 
assignment with a relatively small sample. At the first posttest, the percentage of adults 
with a GED was about the same in the two groups-32 percent in Even Start and 31 
percent in the control group. By the second posttest, a significantly greater percentage 
of adults in Even Start (40 percent) had a GED or diploma compared with the control 
group (33 percent). It can be seen that the percentage of adults who gained the GED 
credential increased steadily among adults in Even Start over the eighteen months of the 
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Exhibit 9.9: Percentage of Adults Attaining a QED 
cr Diploma: In-Depth Study and NEIS 



see 



5.7%:; 



|14.1 % 



IDS Even Start Group IDS Control Group 



NEIS Overall 



NEIS Restricted 



Exhibit reads: Across projects participating in the evaluation, 8.3 percent of the adults who participated in Even 
Start attained a GED or diploma. 
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Exhibit 9.1 0: Percentage of Adults with a High School 
Diploma or GED: In-Depth Study 
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Exhibit reads: Among Even Start adults, 39.5 had a GED or high school diploma by the second posttest. 



q Even Start 

ERIC 

EJM,HJWWB;iHIJilU 



194 ( ' 



Effects of Even Start 
Projects on Parent Literacy 



In-Depth Study. In contrast, the percentage of adults in the control group changes only 
slightly from the pretest through the second posttest. 

Taken together, these results point to strong, positive effects of Even Start on attainment 
of a GED. As Exhibit 9.4 indicated, adults in Even Start are much more likely to be 
enrolled in a GED program than adults in the control group, giving further credence to the 
interpretation that the differences can be attributed to Even Start participation. 

Effects as Measured by the NEIS. Data from the NEIS show that, across all Even Start 
projects participating in the evaluation, 8.3 percent of adults who entered Even Start 
without a GED or diploma achieved one while participating for a year or less in adult 
education services (Exhibit 9.1 1). However, attaining a GED is not a reasonable short- 
term goal for some adults in Even Start. For example, it is unlikely that an adult who 
enters Even Start with a sixth-grade education will be able to achieve a GED in the 
relatively short time frame of this study. Therefore, we conducted another analysis in 
which we restricted the sample of adults to those who entered with at least a ninth-grade 
education and who participated in Even Start for at least three months. By eliminating 
adults who entered with very low education levels or who received very small amounts 
of adult education, this restricted sample contains adults who ought to have had a 
reasonable chance of attaining a GED while in Even Start. As expected, the percentage 
of adults attaining a GED is higher in the restricted sample than in the overall sample 
(14.1 percent vs. 8.3 percent). 

Variation in GED Attainment by Family Characteristics. Additional information about 
attainment of a GED in the NEIS data set is presented in Exhibit 9.11. There are no 
surprises in the exhibit, which shows that several variables are related to attaining a GED. 
As would be expected, attaining a GED during the year is related to grade at intake to 
Even Start: no adults who entered with less than a fifth-grade education attained a GED, 
compared to 10.2 percent of the adults who entered with some high school education. 
Attaining a GED also is strongly related to the adult's functional level on the CASAS: 
adults who scored at the "Beginning," "Basic," or "Intermediate" levels on the CASAS 
pretest rarely attained a GED (2.1 percent), while 16.1 percent of adults who scored at 
the "High School" level on the CASAS pretest attained a GED during the year. Total 
hours of adult education instruction is another variable that is related to attaining a GED: 
adults who had higher amounts of instruction were more likely to attain a GED than adults 
with lower amounts of instruction. High rates of GED attainment also were associated 
with being young, with having a relatively high annual income, with having English as 
one's primary language, and with being white, African-American, or Native American as 
opposed to Hispanic or Asian (this latter finding is likely related to the English language 
problems experienced by Hispanic and Asian immigrants). 

Variation in GED Attainment Across Projects. Given the great variation among projects 
in the characteristics of adults served and the amount of adult education instruction 
received by participating adults, we expect to see substantial variation among projects in 
GED attainment. Exhibit 9.12 shows a distribution of the percent of adults attaining a 
GED for projects in the evaluation. Only adults entering Even Start with some high school 
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Exhibit 9.11 










Number and Percent of Participating Adults 
Who Attained a GED While Served by Even Start (NEIS Data Set) 






All adults lacking a 
diploma at intake 


Adults entering with at 
least a ninth grade 
education and participating 
more than three months 


Group 


Total 


Attained GED 


Total 


Attained GED 




Adults 


N 


% 


A Hi ill"© 


N 


% 


Highest grade at intake 














Grade 0-4 


835 


0 


0.0 


— 






Grade 5-8 


2,862 


146 


5.1 


- 






Grade 9-12 


8,784 


893 


10.2 


5,172 


729 


14.1 


Functional level on CASAS at pretest 










Beginning to 
Intermediate 


2,335 


48 


2.1 


1,054 


35 


3.3 


High School 


4,564 


734 


16.1 


2,663 


532 


20.0 


Total hours of adult education instruction 










1-69 hours 


6,693 


414 


6.2 


2,266 


233 


10.3 


70-200 hours 


2,858 


328 


11.5 


1,746 


255 


14.6 


> 200 hours 


1,720 


280 


16.3 


1,160 


241 


20.8 


Age at intake 














16-21 


1,647 


166 


10.1 


699 


112 


16.0 


21-25 


3,766 


392 


10.4 


1,666 


283 


17.0 


26-30 


3,235 


254 


6.6 


1,462 


185 


12.7 


31-35 


1,768 


116 


5.6 


710 


81 


1 1 .4 


36-40 


862 


48 


5.6 


293 


31 


10.6 


Over 40 


664 


26 


3.9 


195 


18 


9.2 
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Exhibit 9.11 
(continued) 










Number and Percent of Participating Adults 
Who Attained a GED While Served by Even Start (NEIS Data Set) 






Ail adults lacking a 
diploma at intake 


Adults entering with at 
least a ninth grade 
education and participating 
more than three months 


Group 


Total 


Attained GED 


Trttnl 


Attained GED 




Adults 


N 


% 


Adults 


N 


% 


Family annual income 














Under $5,000 


4,919 


352 


7.2 


2,146 


258 


12.0 


$5,000-9,999 


3,540 


281 


7.9 


1,398 


203 


14.5 


$10,000-14,999 


1,885 


162 


8.6 


715 


112 


15.7 


$15,000-19,999 


828 


86 


10.4 


352 


60 


17.0 


$20,000-24,999 


394 


62 


15.7 


190 


39 


20.5 


Over $25,000 


305 


49 


16.1 


138 


29 


21.0 


Primary language is English 












Yes 


8,773 


948 


10.8 


4,214 


661 


15.7 


No 


3,461 


82 


2.4 


866 


59 


6.8 


Gender 














Male 


1,025 


81 


7.9 


404 


46 


11.4 


Female 


11,009 


936 


8.5 


4,600 


669 


14.5 


Ethnic background 














Asian 


311 


9 


2.9 


72 


7 


9.7 


African-American 


3,848 


188 


4.9 


1,805 


138 


7.6 


Hispanic 


3,359 


97 


2.9 


885 


75 


8.5 


Native American 


631 


64 


10.1 


307 


42 


13.7 


White 


4,101 


676 


16.5 


2,018 


462 


22.9 


TOTAL 


12,481 


1,039 


8.3 


5,172 


729 


14.1 


Exhilvt reads: Among the 12,481 adults lacking n high school diploma or GED at intake, 8.3 percent 
attained a GED while participating in Even Start. 
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Exhibit 9.12: Project-Level Distribution of the 
Percentage of Adults Who Attained a GED (NEIS Data Set) 




Percent of Adults Attaining GED 



Exhibit reads: Less than five percent of the adults attained a GED in 53 projects while 50 percent attained a 
GED in one project. Only adults with nine to 12 years of schooling at intake and at least three months 
participation in adult education were included in the analysis. 

education and at least three months of adult education instruction were included. 
Twenty-three projects had fewer than ten adults meeting these criteria, and hence were 
excluded from the analysis. Among the remaining 100 projects, the within-project 
percentage of adults attaining a GED ranged from zero to 60 percent. Over three-quarters 
of the projects (77) had less than ten percent of their adults attaining a GED while one 
project reported that more than 50 percent attained a GED. 



Reading and Writing Activities in the Home 

Two sets of questions were included in the In-Depth Study parent interview to ask 
parents about their own reading and writing habits in the home (these questions were not 
included in the NEIS portion of the evaluation). Both lists were adapted from a 
questionnaire used to evaluate the California State Library's Adult Learner Program. 



Description of the Measure 

The 13 self -report questions about reading activities ask how often the parent reads 
different types of literacy materials commonly found in and around the home including 
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junk mail, letters and bills, coupons, labels on food, cooking recipes, religious materials, 
instructions, street signs, newspapers, notes from school, T.V. Guide , magazines, and 
books. The 1 1 self-report questions about writing activities ask how often the parent 
writes at home in the following areas: checks, notes, recipes, forms or applications, 
dates on a calendar, letters, stories or poems, greeting cards, crossword puzzles, grocery 
lists, and a journal or diary. 

These questions are intended to give an indication of the literacy environment in the 
home. Since most Even Start parents are expected to be poor readers, it is not realistic 
to expect them to be avid readers and writers. However, it is not known how much they- 
use reading and writing in common household activities. If few of these activities are 
undertaken, then these homes would truly be impoverished literacy environments. On the 
other hand, an increase in the use of these simple literacy tasks would be a realistic goal 
for adults who are working to improve poor reading skills. 

Parents responded to each of the questions on a three-point rating scale where a value 
of 1 meant that the activity was done "not at all," a value of 2 meant that the activity 
was done "sometimes," and a value of 3 meant that the activity was done "regularly." 
The composite score for a given parent was computed as the average rating for all 
completed items in the scale-higher scores mean a higher level of reading or writing 
activities in the home. Thus, the highest possible composite score is 3.0, and the lowest 
possible score is 1 .0. The reliability (internal consistency) of both scales as assessed in 
this study is quite reasonable: .80 for the reading activities scale and .71 for the writing 
activities scale. 



Effects on Reading and Writing Activities 

Exhibits 9.13 and 9.14 summarize In-Depth Study analysis results for the Reading 
Activities scale and the Writing Activities scale. Parents in Even Start had a pretest mean 
of 2.2 points on the Reading Activities scale while parents in the control group had a 
pretest mean of 2.1 points. On the Writing Activities scale, parents in Even Start had a 
pretest mean of 1 .8 points, while control group parents had a pretest mean of 1 .7 points. 

Thus, parents in each of the two groups scored in the middle of the scale range on the 
pretest, indicating that they do not report particularly low levels of reading and writing 
activities in the home. While these pretest levels may be higher than expected, there still 
is room for growth on the posttest. Exhibit 9.1 3 shows slight gains for the control group 
on ratings of reading activities between pretest and the second posttest. Exhibit 9.14 
shows similarly small gains favoring the program group on writing activities. However, 
there are no significant program effects on either variable at the second posttest. 
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Exhibit 9.13 








Parent Reading Activities in the Home: 
Effects from the In-Depth Study 








Even Start 
(n = 84) 


Control 
(n = 74) 






Mean 


S.D. 


Mean |j 


S.D. 


Pretest 




2.2 


0.4 


2.1 


0.4 


Second Posttest 




2.2 


0.4 


2.2 


0.3 


Gain 




0.03 




.07* 




Program Effect 
(Effect Size) 






-0.04 points 
(.10 s.d.) 




• p < .05 












Exhibit reads: Even Start adults in the in-Depth Study averaged 2.2 points on Reading 
Activities in the heme pretest. 

Note: Scores range from 1 (no reading at all) to 3 (reading regularly). 









Exhibit 9.14 










Parent Writing Activities in the Home: 
Effects from the In-Depth Study 








Even Start 
(n = 84) 


Control 
(n=74) 






Mean || S.D. 


Mean 


S.D. 


Pretest 




1.8 0.3 




1.7 


0.3 


Second Posttest 




1.9 0.4 




1.8 


0.3 


Gain 




0.12* 




0.06 




Program Effect 
(Effect Size) 




.06 points 
(.20 s.d.) 




• p < .05 












Exhibit reads: Even Start adults in the In-Depth Study averaged 1.8 points on Writing Activities 
in the home at pretest. 

Note: Scores range from 1 (no writing at all) to 3 (writing regularly). 
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This evaluation measured effects on parent literacy skills in three areas: (1) functional 
literacy levels on a reading test, (2) the pe r centage of Even Start adults who obtained 
their GED certificate, and (3) reading and writing activities in the home. Even Start has 
had clear positive effects on GED attainment; the data are positive but mixed about 
effects on functional literacy; and there is no evidence that Even Start has changed 
reading and writing activities as reported by parents. 

The results for GED attainment are strong and persuasive. Data from both the In-Depth 
Study and the NEIS lead to the same conclusion: that Even Start has led to a substantial 
increase in the percentage of adults attaining a GED. Without Even Start, it is likely that 
few of these adults would have found the needed assistance to reach this goal, as 
evidenced by the In-Depth study data on participation in GED classes. 

Even Start projects may have been effective in improving the functional literacy of 
participating adults. Data from the NEIS show that adults who participate in Even Start 
achieve positive gains on the CASAS Reading Survey, gains which are comparable or 
greater in size than those observed in other studies of adult education programs. 
However, data from the In-Depth Study show that the gains of Even Start adults are not 
significantly greater than the gains achieved by a randomly assigned control group. 

Data from the NEIS show that the amount of gain on the CASAS is directly related to the 
amount of instruction received through Even Start. This finding suggests that the 
observed gains can be attributed to adult education. However, this hypothesis is not 
necessarily supported by the In-Depth Study data. In this small randomized study we find 
that adults in Even Start are more likely to participate in adult basic education and GED 
preparation than adults in the control group, yet this does not appear to result in increased 
CASAS scores. These results call into question the sensitivity of the CASAS to 
sometypes of instruction. The CASAS gains seen among the control group also suggest 
that gains on the CASAS may be expected in the absence of formal instruction. 

Finally, we were not able to detect any measurable program effects over an eighteen- 
month period on the extent to which parents use reading and writing as literacy tools in 
the home. 
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Chapter Ten 

Effects of Even Start Projects on Parenting 
Skills 



This section presents findings from the NEIS and the In-Depth Study about Even Start's 
effects on parenting skills. We use the term "parenting skills" to broadly include parents' 
knowledge about appropriate child behaviors and developmental needs as well as the 
activities and materials available in the home to foster children's development. Helping 
parents support their children's growth and development is one of the primary goals of 
Even Start, and the inclusion of parenting skills as a key program component is one of the 
features that distinguishes Even Start from other literacy and adult education programs. 
This section of the report includes discussions of: 

• Adults' Personal Skills 

• Home Learning Environment 

• Parent-Child Reading Task 

• Parents' Expectations for Their Children 

• Conclusions about Effects of Even Start on Parenting Skills 



Adults' Personal Skills 

The parent interview for the In-Depth Study included two existing scales to assess 
parents' depression and sense of self -efficacy or locus of control. These measures were 
not part of the NEIS data collection. 



Description of the Measures 

Parents' personal skills have been found to be important mediating variables for parent- 
child interaction and parents' involvement in their children's education and development 
(Cleary, 1988; McLoyd, 1990; Parker et al., 1988). Parents who are depressed, have 
poor self-esteem, or feel "out-of -control" are not likely to provide optimal conditions for 
their children's growth and development (Upshur, 1988). 

The Pearlin Mastery Scale was used to measure parents' sense of self-efficacy or locus 
of control. This scale assesses the extent to which an individual regards one's life as 
being under one's own control rather than determined by fate. This scale is included in 
the national evaluation of the JOBS program, and was selected over other related scales 
because it includes a mix of positive and negative items and has high internal consistency 
(Cronbach's alpha of .81 in prior research and .63 for the In-Depth Study sample.) The 
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author's work (Pearlin and Schooler, 1978) showing the relationship of the Mastery Scale 
to stresses of parenting and family life provides evidence of the scale's construct validity. 

The scale consists of seven items such as "There is really no way I can solve some of the 
problems I have" and "I can do anything I really set my mind to do." The respondent is 
asked to rate each item on a four-point scale where "4" indicates "strongly agree" and 
"1" represents "strongly disagree." Scores were reversed for those items stated 
negatively, in order to create an average score where "4" indicates a sense of mastery. 

Parents' depression was assessed by the Center for Epidemiologic Studies Depression 
Scale (CES-D) developed by Radloff (1977). This instrument is one of the most frequently 
used measures of depression cited in the psychological literature. The items were 
designed for use in general population surveys as a short self-report measure. The 20 
items on the scale, which represent a subset of those previously validated on longer 
depression scales, have high internal consistency as indicated by a Cronbach's alpha of 
.90 in previous research (Hall et al., 1985) and .89 in the In-Depth Study. Moderate test- 
retest correlations of .51 to .67 have been reported when the scale is administered after 
two to eight weeks (Radloff, 1977). The construct validity of the scale has been 
demonstrated by correlations of .49 to .85 with clinicians' ratings and correlations of .72 
to .84 with longer self-report scales used with psychiatric and general populations 
(Weissman et al., 1977). 

Items on the CES-D include statements such as "I felt that everything I did was an effort," 
"I had crying spells," and "I enjoyed life." Respondents were asked to indicate how often 
they experienced each feeling during the past week. Items are rated on a four-point scale 
where "0" indicates "rarely or none of the time -- less than one day" and "3" reflects 
"most or all of the time - 5-7 days." Positive items were reversed so that a total score 
could be computed, which ranges from 0 to 60. A total score of 16 or higher is 
considered to be indicative of high depressive symptoms (Hall et al., 1985). 



Effects on Personal Skills 

Data from the In-Depth Study showed no significant program effects on parents' personal 
skills, although the prevalence of depressive symptoms among adults in both Even Start 
and thn control group is noteworthy. 

Effects as Measured by the In-Depth Study. Exhibit 10.1 presents pretest scores and 
gains at the second posttest on the Pearlin Mastery Scale and CES-D for Even Start and 
control parents. There are no significant program effects on either measure. For the CES- 
D, both the average total score and the percentage of adults with high depressive 
symptoms are shown. At the In-Depth Study pretest, 46 percent of Even Start parents 
and 39 percent of control parents had scores above 16, indicating high depressive 
symptoms. By the second posttest, both groups improved slightly. However, there is no 
significant difference in the gains for Even Start compared to gains for the control group. 
Depression levels have proven difficult to change with this type of program. The national 
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Exhibit 10,1 








Effects on Parents' Personal Skills: 
Results from the In-Depth Study 








Euan Ctflrt (n = 


= 80) 


Control ( n = 

vwl • M Ul \ II 


71) 




Parenting 
Variable 


Pretest Mean 
(s.d.) 


Gain 


Pretest Mean 
(s.d.) 


Gain 


riuyitsiM c tic u i 

(Effect Size) 


Pearlin 
Mastery" 


2.9 (0.6) 


0.01 


2.9 (0.6) 


0.22* 


-0.21 (.36 
s.d.) 


CES-D Total" 


15.0 (11.6) 


0.16 


15.0 (12.5) 


-0.58 


0.74 (.06 s.d.) 


High 

Depressive 
Symptoms' 1 


46.3% (50.1%) 


-2.5% 


39.4% (49.0%) 


-4.2% 


1.7% (.03 
s.d.) 


Exhibit reads: At pretest, 46.3 percent of Even Start perents reported high depressive symptoms. 


•Score represents average across seven items rated on four-point scale, where "4" indicates more positive behavior or 
attitudes. 

'Scores represent frequency of 20 depressive symptoms rated on three-point scales. 
"Percentages represent proportion of parents scoring 1 6 or higher on CES-D total. 



CCDP evaluation (St. Pierre, et al., 1994) reported no significant effect of the program 
using the same measure of depressive symptoms. 

In spite of the absence of program effects, it is interesting to note the proportion of adults 
in both groups who report depressive symptoms. These results are similar to those 
reported by Hall and her colleagues (1985), who found that 48 percent of low-income 
mothers of young children had scores above the cut-off for the CES-D. 



Home Learning Environment 

The NEIS instruments and the In-Depth Study parent interview each include several 
questions about parents' interactions with their children at home and the extent to which 
the home environment is conducive to young children's growth and development. 



Description of the Measures 

Information about parenting and the home environment comes from in-person parent 
interviews collected as part of the NEIS and the In-Depth Study. The questions were 
drawn from three existing instruments: 
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• Home Observation for Measurement of the Environment (HOME). 
The HOME Inventory (Caldwell & Bradley, 1984) has been used to 
measure the impact of parent training and education and to assess 
the quality of the home environment and mother-child relationship in 
the National Longitudinal Study of Youth (NLSY). Questions on 
reading activities and toys parents provide for learning were adapted 
for the Even Start evaluation. 

• High/Scope Home Environment Scale (H/SHES). A structured parent 
interview was developed by the High/Scope Educational Research 
Foundation to assess parenting and parent-child interactions for the 
National Home Start evaluation (High/Scope Educational Research 
Foundation and Abt Associates Inc., 1975). Questions were drawn 
from this instrument for Even Start in four categories of activities: 
books and reading, play materials available in the home, pa-ent 
teaching, and participating in learning activities. 

• Parent as a Teacher (PAAT). The Parent as a Teacher (PAAT) self- 
rating scale was developed by Strom (1984) to assess parents' 
attitudes toward their children and to determine feelings and values 
concerning children's behaviors. This instrument was used in the 
evaluation of Project Giant Step (Layzer, Goodson, & Layzer, 1991) 
in New York City, a preschool program for disadvantaged families. 
For the NEIS, questions were selected from two subscales: play -- 
understanding the developmental function of play and willingness to 
participate in play with the child, and teaching -- understanding the 
learning process in young children and confidence in the parent's role 
as a teacher. For the In-Depth Study, two subscales were added: 
creativity -- parents' accept creative functioning and encourage its 
development, and frustration -- absence of frustration or irritation 
with child's demand for attention. 

The parent interview developed for the In-Depth Study was purposefully designed to 
include the same questions as the NEIS while adding questions about parent-child 
activities outside of the home and family rules. These items were added because 
research with low-income families (e.g., Snow et al., 1991) suggests that homes 
associated with poor school performance often lack structure and supervision. In 
addition, children from low-achieving homes frequently lack the opportunities that middle- 
class children have for experiences and interactions outside of their families that can 
afford opportunities for learning (Cochran and Brassard, 1979). 

The home environment items on the parent interviews ask parents either to indicate the 
frequency of their own or their child's behaviors or to rate their agreement with 
statements about child development. In most cases, there are several behaviors or 
statements for each construct that v/e are interested in measuring. For example, the 
question about play materials in the home lists 1 2 toys or materials and parents are asked 
to indicate which ones are available in their home. For the analyses, composite scores 
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were used rather than individual items, in order to handle missing data and increase the 
reliability of the scores. For items scored on three- or five-point scales, composite scores 
were based on an average across individual item responses. For items scored yes or no, 
composite scores reflect the percentage of positive responses. 

Exhibit 10.2 describes these composite scores and presents the internal consistency of 
the composite scores from the NEIS and the In-Depth Study data. The internal 
consistency, based on Cronbach's alpha coefficient, is a measure of the extent to which 
responses to individual items are correlated to the total score. Internal consistencies of 
.60 or .70 are considered to be moderate, and coefficients above .80 indicate a highly 
reliable composite score. In general, the reliabilities for the In-Depth Study composites 
are lower than those for the NEIS because of smaller sample of respondents. 

Nevertheless, most of the coefficients exceed .60, indicating moderate to good internal 
consistency. 

Two individual items about the home environment also are included in the parent 
interviews from the NEIS and the In-Depth otudy. These include: 

• Story reading: the frequency with which the parent reads to their 
child, rated on a five-point scale from never (1) to every day (5). 

• Books in the home: the number of books in the home for the child to 
look at or read, categorized as none, one-two books, three-nine 
books, or more than ten books. 



Effects on Home Learning Environment 

Overall, data from the NEIS and In-Depth Study point to few significant program effects 
in this area. Data from the NEIS show statistically significant gains on most of these 
measures; however, the effect sizes are generally small. In the in-Depth Study, program 
families show gains on some measures, but gains are also seen among control group 
families. 

Effects as Measured in the In-Depth Study. Exhibit 1 0.3 presents the results from the In- 
Depth Study on the home environment measures. The one statistically significant 
program effect is seen for the number of reading materials in the home. This variable 
measures the number of different types of reading materials in the home (e.g., books, 
magazines, newspapers). Even Start families show a 17 percent gain from pretest to 
second posttest, which : s a statistically significant difference. In contrast, control families 
exhibit a six percent increase, which is not significant. The larger gain for Even Start 
families translates into a statistically significant program effect that corresponds to an 
effect size of .40 standard deviation units. This is a moderate gain in educational or 
social programs. 
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Exhibit 10.2 

Description and Reliability of Home 
Environment Composite Scores 


Composite 
Score 


Description 


Response Choices 


Internal 
Consistency* 


NEIS 


IDS 


Learning 
activities 


Activities that can be used to 
teach or reinforce cognitive 
skills such as finding food on 
store shelves or sorting clean 
clothes (5 items on NEIS; 9 on 
IDS) 


1 Never 

2 Once or twice 

3 On a regular basis' 1 


.73 


.67 


Reading 
materials 


Reading materials found in the 
home (5 items) such as 
magazines and newspapers 


1 No 

2 Yes 


.72 


.50 


Play 

materials 


Materials found in the home 
that children can play with such 
as scissors, paints or puzzles 
(12 items on NEIS, 13 on IDS) 


1 No 

2 Yes 


.77 


.68 


Teaching 
child 


Things parent has helped child 
learn during the past month 
such as nursery rhymes and 
colors (11 items on NEIS, 7 on 
IDS) 


1 No, did not help 

2 Yes, helped 


.91 


.61 


Talk with 
child 


Topics parent have discussed 
with child (8 items) such as 
school activities or a television 
program 


1 Never 

2 Rarely, if ever 

3 Once/twice a month 

4 Once/twice a week 

5 Daily 


.93' 


.65 


Family rules 


Extent to which family has rules 
for child's behavior (7 items) 
such as amount of television or 
helping with chores 


1 Not at all liko family 

2 Somewhat like family 

3 Very much like family 


NA 


.68 


Activities 
with child 


Parent-child activities outside of 
home in the past month (7 
items) such as going to a park 
or grocery store 


1 Not at all 

2 Once or twice 

3 Once a week 

4 Several times a week 

5 Almost every day 


NA 


.49 


Parent as a 
Teacher 


Agreement with statements 
that reflect how children learn 
or the parent's role as a teacher 
(14 items on NEIS; 28 in IDS) 


1 Disagree strongly 

2 Disagree somewhat 

3 Agree somewhat 

4 Agree strongly 


.70 


.74 


'Cronbach's alpha ranges from 0 to 1.0 and indicates the relationship between individual items and composite 
scores. 

'Response choices were extended to a five-point scale on IDS measure: never, once or twice, at least once/weBk, 
2-3 times per week and daily. 
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Exhibit 10.3 








Effects on the Home Learning Environment: 
Results from the In-Depth Study 








Even Start (n = 


: 84) 


Control ( n = 


73) 




Home Environment 
Variable 


Pretest Mean 
(s.d.) 


Gain 


Pretest Mean 
(s.d.) 


Gain 


Program Effect 


Learning activities- 1 


3.4 (.72) 


.12 


3.7 (.76) 


.23* 


-.10 (.14 s.d.) 


Story reading* 


3.5 (1.2) 


-.07 


3.3 (1.2) 


-.17 


.10 (.08 s.d.) 


Books in home* 


3.4 (.74) 


.13 


3.2 (.85) 


.26* 


■ -.13 (.15 s.d.) 


Reading materials' 1 


54.2% (23.5) 


16.9%* 


55.8% (26.2) 


6.4% 


10.5%* (.40 s.d.) 


Play materials" 


58.0% (18.0) 


1 1 .5%* 


56.2% (19.6) 


9.3%* 


2.2% (.1 1 s.d.) 


Teaching child 6 


73.1% (22.6) 


5.5% 


69.2% (20.9) 


4.4% 


1.1% (.05 s.d.) 


Talk with child* 


3.8 (.64) 


.15* 


3.7 (.76) 


.24* 


-.09 (.12 s.d.) 


Parent as a Teacher* 


3.1 (.34) 


.03 


3.0 (.36) 


.02 


.01 (.03 s.d.) 


Family rules" 


2.5 (.35) 


.19* 


2.7 (.38) 


.06 


.13 (.34 s.d.) 


Activities* 


2.3 (.48) 


.01 


2.2 (.46) 


.1 1 


-.09 (.19 s.d.) 


Exhibit reads: At pretest, Even Stait parents reported having 58 percent of the play materials listed on the parent interview. 


* p < .05 

'Based on three- or five-point scale. 

'Score reflects percentage of materials or activities for child. 









There are statistically significant differences between the pretest and second posttest for 
both Even Start and control group parents on a few other variables. For example, on play 
materials in the home and talking with child at home the Even Start and control groups 
each gain a significant amount from pretest to second posttest. Because both groups 
gain roughly comparable amounts, there are not any statistically significant program 
effects for these variables. 

Effects as Measured by the NEIS. Exhibit 10.4 presents the results on the home 
environment measures from the NEIS. The gain from pretest to posttest is statistically 
significant for all of these measures. This is partly due to the large number of adults in 
the NEIS database. When the effect sizes are considered, the gains range from small to 
moderate in size. 

In order to explore the relationship between gains seen on these parenting measures and 
participation in Even Start parent education, gains were calculated separately for three 
levels of parent education attendance, controlling for pretest scores on the parenting 
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Exhibit 10.4 






Effects on the Home Learning Environment: 
Results from the NEIS 




Measure 


Number 
of Adults 


Average 
Pretest 


otanoara 
Deviation 


Gain 


Effect 


Learning 
activities 0 


4,401 


2.0 


0.59* 


.27* 


.46 


Story reading" 


4,379 


3.4 


1.25 


.36* 


.28 


Books in home 8 

UX/vl\w 111 I 1 v 1 1 


4,372 


3.4 


0.87 


.27* 


.31 


Reading 
materials b 


4,129 


48.2% 


26.1 


6%* 


.23 


Play materials' 3 


4,164 


54.2% 


23.8 


15%* 


.63 


Teaching child b 


4,062 


67.5% 


26.7 


12%* 


.44 


Talk with child" 


4,352 


2.9 


1.34 


.54* 


.40 


Parent as a 
Teacher" 


4,213 


3.3 


0.39 


.07* 


.02 


Exhibit reads: Even Start parents scored 2.0 at pretest on the Learning Activities scale. 


* p < .001 . 

"Based on three- or five-point scale. 

b Score reflects percentage of materials or activities for child. 







measures and the age of the child (Exhibit 10.5). The gains for adults who participated 
in more parenting education are significantly higher, but not importantly larger, than for 
parents with more minimal participation levels on all indicators. Exhibit 10.5 also shows 
that the effect sizes for these indicators are small to moderate for all except the "play 
materials" scale. 

Discussion of Effects on the Home Learning Environment. The Even Start effects we 
observe on the home environment measures are small at best. There are several reasons 
why these findings may be weaker than those observed in adult education or early 
childhood education. First, there are few models to guide Even Start projects in 
developing effective instructional approaches to improving parenting skills. Second, the 
data shown earlier in this report indicate that, on average, participants receive much less 
parenting education than literacy training. The research literature concurs that the 
outcomes of parenting education are particularly difficult to assess and there is little 
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Exhibit 10.5 










Gains on Home Environment Measures in the NEIS 






by Hours of Parenting Education 








Hours of 


Number of 


Average 


Average 


Effort 




parenting 


Adults with 


Pretest 


Gain 


OIZc 




Education 


Matched 












Scores 








Learning 


1-50 hours 


2,553 


1.99 


.24 


.40 


ul/llvllICO 


51-150 hours 


1,351 


2.01 


"5 1 
.O I 


.□O 




over 1 50 hours 


51 1 


2.03 


.31 


.53 


Story reading 


1-50 hours 


2,553 


3.38 


.33* 


.26 




51-1 50 hours 


1,339 


3.55 




97 




Over 1 50 hours 


508 


3.48 


.44* 


.37 


books in 


1-50 hours 


2,544 


3.34 


.27* 


.31 


hnrmp 

1 IUI 1 IC 


51-150 hours 


1,336 


3.42 








Over 1 50 hours 


507 


4.42 


.32* 


.36 


Reading 


1-50 hours 


2,392 


1 .49 


.05* 


.19 


mptprialc 


51-150 hours 


1,269 


1 .49 


,uo 


.OZ 




Over 1 50 hours 


479 


1.49 


.10* 


.38 


Dim/ 

riay 


i -ou nours 


Z.odi 


1 .be 


.13 


.54 


materials 


51-150 hours 


1,292 


1.55 


16* 


R7 




Over 1 50 hours 


499 


1.55 


.18* 


.78 


Toll/ - lAiitk 

i am wren 


1-50 hours 


2,531 


2.94 


.43* 


.31 


child 


51-150 hours 


1,343 


2.93 


.77* 


fiO 




Over 1 50 hours 


510 


3.14 


.78* 


.63 


Parent as a 


1-50 hours 


2,448 


3.33 


.05* 


.13 


Teacher 


51-150 hours 


1,288 


3.35 


.08* 


.20 




Over 1 50 hours 


493 


3.31 


.15* 


.37 


Exhibit reads: Parents with more than 150 hours of parenting education scored 3.48 on the story 


reading item, compared with parents with 1-50 hours of parenting education who scored 3.38. 


* p < .001. 
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consensus about the appropriate constructs and few psychometrically sound measures 
(Weiss and Jacobs, 1988). 

A further consideration is that high pretest averages on some of the measures leave little 
opportunity to show growth. Parents in both the Even Start and control groups generally 
reported on the pretest that their child had participated in many learning activities at 
home, that there were many books in the home, and that they had done many things to 
help their child learn. It is possible that parents were trying to give socially acceptable 
answers and that the "right" answers to items were too transparent. 



Parent-Child Reading Task 

A Parent-Child Reading Task was designed specifically for the In-Depth Study to measure 
the parent-child interactions of Even Start participants and control families. The task 
consists of asking the parent to read a simple book to her child, while a trained observer 
uses a pre-coded rating form to record several aspects of parent-child interactions. The 
observation was developed to provide a direct measure of a shared literacy activity and 
parent-child interactions. 



Description of the Measure 

The task is based on research that stresses the importance of reading books for the 
literacy development of young children. Findings from various studies over the last two 
decades note that middle-class children who are read to understand and know more 
words (Chomsky, 1972), have increased language and reading growth (Ninic and Bruner, 
1978), learn basic concepts about books (Sulzby, 1985; Snow and Ninio, 1986), and 
benefit additionally because adults' reading to children acts as "scaffolds" in emerging 
literacy development (Edwards, 1989). Other research documents the importance of the 
methods used by parents when they read aloud to their children. The manner in which 
parents read to children can inhibit or facilitate the child's interest and skill in reading 
(Lancy, 1988). The quality of the interaction surrounding parent-child reading has been 
found to shape early reading development more than merely the presence of books or 
storytime routine (Mason and Allen, 1986). 

Further, parents' questioning strategies used when reading aloud to children may promote 
or limit the cognitive benefits of story book reading (Pellegrini etal., 1990). Heath (1983) 
has written of the importance that questioning plays in the development of school-type 
literacies in young children. In three communities she studied, only parents in the middle- 
class sample used the kind of cognitive questioning strategies that teachers commonly 
use in school-based instruction. She considers children who are not familiar with this 
cognitive approach to be at risk when confronted with the demands of formal schooling. 
Her sample included poor black families from a rural southern region, not unlike some of 
the Even Start sites. 
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Thus, the research indicates that one way that parents teach children is through the 
medium of story book reading as they question and comment on the text and pictures, 
and as they initiate and respond to the child's comments. The management of the story 
book session reveals much about the relationship, interaction, and teaching strategies that 
occur between the parent and child around a pleasurable task which, if effectively carried 
out, promotes literacy development. 

Several criteria were used to select tru book for this task: 

• The book had to have a story line. Previous research indicates 
reading stories has stronger associations with children's reading and 
language development than looking at books or reading wordless 
picture books, alphabet books or nursery rhymes (Wells, 1985). 

• The book had to be available in Spanish and English to ensure 
comparability of subject and difficulty of the story across language 
groups. 

• The book had to have relatively easy vocabulary so that it would not 
be too difficult for low-literate parents. 

• The book had to have based on subject matter appropriate for girls 
and boys, children from diverse cultures, and children living in urban 
and rural areas. 

A book entitled Three Billy Goats Gruff was selected. It met all of the above criteria. 

Parents were asked to read the book to their child during the testing/interview session for 
the In-Depth Study. The choice of the English or Spanish version of the book was left up 
to the parent. The interviewer told the parent that we were interested in learning how 
parents and children read together, and instructed the parent to read the book the way 
she usually would read to her child. The interviewer rated the parent's and child's 
behavior as they read the book. 

The rating scale was developed to record parent-child interactions and the parent's 
approach to reading during the brief story book reading session. The rating scale is 
divided into two parts. On Part I, the interviewer scored the way the parent read the 
book (e.g., labeling pictures, asking questions of the child) as well as the child's response 
and behavior during the session (e.g., pointed to pictures, responded to parent's 
questions). These categories are adapted from a rating scale developed by Resnick and 
his colleagues (1987), based on videotapes of mothers reading to their young children, 
and modified by Edwards (1989). The items are scored as absent or present; counts of 
behavior were not recorded because pilot testing of the instrument indicated it was not 
feasible to categorize and keep a count of diverse behaviors during these relatively brief 
observations. 
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Part il of the coding scheme was filled out by the interviewer at the completion of the 
parent-child reading activity. These general ratings of the quality of the parent-child 
interaction and the parent's reading are adapted from work conducted by Lancy and 
Draper (1988). This section includes items rated on a three-point scale to describe the 
reciprocity and quality of parent-child interaction. Parent and child were rated separately 
on items such as physical contact and task engagement. 

Interviewers were trained by using videotapes of several parent-child dyads reading Three 
Billy Goats Gruff. In addition, an administration manual provided a description and 
examples of each coding category. For the analysis, composite scores were created as 
more reliable indicators. Based on conceptual groupings of parents' behaviors assessed 
on Part I, two composite scores were created and then evaluated for internal consistency. 
Additional composite scores were created by aggregating child behaviors and the general 
affect scores. The composite scores from Part I of the rating scheme were created by 
totaling the number of different behaviors that occurred during the session. On Part II, 
composite scores were calculated as the average scoring across the three-point scales. 
Exhibit 10.6 presents the reliability coefficients for the composite scores and lists the 
individual items included. Reliabilities are quite good, ranging from .71 to .80 for the five 
composite scores. 



Effects on the Parent-Child Reading Task 

There are no significant gains for any of the indicators of parents' or children's behaviors 
in either Even Start or the control group, and no program effects. 

Effects as Measured by the In-Depth Study. The pretest means and gains for the 
behavioral ratings and general affect ratings on the Parent-Child Reading Task are 
presented in Exhibit 10.7. Parents and children in both Even Start and the control groups 
tended to be rated lower at the second posttest than at pretest. These consistent 
negative results across groups call into question the utility of this new measure. 



Parents* Expectations for their Children 

Description of the Measures 

Questions about parents' expectations for their children's school success were adapted 
from the parent interview designed by Abt Associates for use in Prospects, the national 
longitudinal study of Chapter 1 (Puma et al., 1 993). In both the NEIS and In-Depth Study, 
parents were asked to rate how well they expected their child to do in school (from "very 
well" to "very poorly") and the likelihood that their child will graduate high school (from 
"very likely" to "probably won't graduate high school"). 
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Exhibit 10.6 




Composition and Reliability of Scores on Parent-Child Reading Task 


Composite Score 


Items 


Internal 
Consistency 
Coefficients 


Parent describes book to 
child 8 


Points to pictures or words; labels 
pictures; describes pictures or 
text; elaborates on pictures or 
text. 


.71 


Parent questions or 
responds to child 9 


Links pictures or story to child's 
life; asks child to identify pictures; 
repeats child's comments or 
words; elaborates on child's 
comments; responds to child's 
questions; asks comparative, 
inferential, or cause-and-effect 
questions. 


.71 


Child responds" 


Points to pictures; labels or names 
picture; repeat words or elements 
of story; acts out or makes sound 
related to story; links story 
content or pictures to own life; 
responds to parent s questions; 
asks questions, makes comments 
related to book or parent's 
comments. 


.74 


Parent's general affect" 


Task engagement; positive affect; 
interactive contact with child; 
physical contact with child; pace; 
control of book; reading fluency; 
reading delivery. 


.80 


Child's general affect" 


Task engagement; positive affect; 
interactive contact with parent; 
physical contact with parent. 


.75 


'Each item in composite scored as present or absent. 
"Each item in composite scored on three-point scale. 
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Exhibit 10.7 



Effects on Parent-Child Reading Behaviors: 
Results from the In-Depth Study 



Parent-Child 
Reading Behaviors 


Even Start (n = 
79) 


Control ( n = 


= 71) 


Program 
Effect (Effect 
Size) 


Pretest 
Mean 

(s.d.) 


Gflin 


Pretest Mean 

(s.d.) 


Gain 


Parent describes book* 


2.1 (1.4) 




2.4 (1.4) 


-0.28 


-0.02 (.01 






0.30 






s.d.) 


Parent questions or 


1.6 (1.6) 




1.9 (1.6) 


-0.49 


-0.08 (.05 


responds to child" 




0.57 






s.d.) 


Child responds' 


2.7 (2.0) 




2.9 (2.0) 


-0.15 


-0.15 (.08 






0.30 






s.d.) 


Parent's general affect" 


2.3 (0.40) 




2.4 (0.39) 


-0.03 


-0.01 (.03 






0.04 






s.d.) 


Child's general affect' 


2.1 (0.50) 


0.08 


2.3 (0.50) 


0.08 


0.0 (--) 



Exhibit reads: Evan Start parents at pretest exhibited en evarege of 2.1 out of 4 behaviors related to describing 

the book to their child. 
'Score represents occurrence of four parent behaviors. 
""Score represents occurrence of six parent behaviors. 
"Score represents occurrence of seven child behaviors. 

"Score represents average of eight items reted on three-point sceles, where "3" indicetes positive behevior. 
'Score represents everage of four items reted on three-point sceles, where "3" indicetes positive behevior. 



Effects on Parents' Expectations 

The NEIS and In-Depth Study data each show significant gains and similar results for 
parents' expectations about their children's school success. However, the control group 
in the In-Depth Study also shows significant gains, and, thus, there is not a significant 
program effect on this variable. 

Effects as Measured by the In-Depth Study. Exhibit 10.8 presents expectations for 
children among parents in Even Start and the control group. The gain seen in the control 
group is statistically significant. However, because the program group was slightly higher 
at the pretest, the two groups are quite comparable by the second posttest. 

Parents in both Even Start and the control group, on average, report that their children are 
likely to graduate from high school. As Exhibit 10.8 indicates, there are no statistically 
significant differences from pretest to second posttest for either group, and no program 
effect. 
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Exhibit 10.8 






Effects on Parent's Expectations for Their Child: 
Results from the In-Depth Study 






Even Start (n 


= 75) 


Control ( n = 


= 58) 




Parent's Expectations 


Pretest 
Mean (s.d.) 


Gain 


Pretest Mean 
(s.d.) 


Gain 


Program 
Effect (Effect 
Size) 


How well child will do 
in school* 


4.3 (0.84) 


0.19 


4.0 (0.82) 


0.34* 


-0.15 (.18 
s.d.) 


Likelihood of child 
graduating from high 
school" 


3.7 (0.57) 


0.06 


3.6 (0.70) 


0.10 


-0.04 (.06 
s.d.) 


Exhibit reads: Even Start parents at pretest rated the likelihood thet their child will graduate from high school as 
3.7 out of a possible 4. 


"Item rated on five-point scale where "5" equals "very well" and "1" indicates "very poorly". 

b ltem rated on four-point scale where "4" indicates "very likely" and "1" indicates "probably will not graduate 

from high 6chool. 



Effects as Measured by the NEIS. The results from the NEIS are presented in Exhibit 
10.9. The average expectations and the magnitude of the gain are almost identical to 
those seen in the IDS. Given the sample sizes here, these small gains are statistically 
significant for the NEIS d"Jta. 



Conclusions about Effects of Even Start on Parenting Skills 

This evaluation measured effects on parenting skills in four areas: (1) parents' personal 
skills, (2) the home learning environment, (3) parent-child reading, and (4) parents' 
expectations for their children. There are few program effects on these measures. 

Parents' personal sense of well-being, as measured by a sense of mastery and lack of 
depression, has been cited in the research literature as related to the nature and quality 
of parent-child and family relationships. Data from the In-Depth Study did not reveal any 
effect of Even Start on these variables. The overall prevalence of depressive symptoms 
was high among this group of families (46 percent of Even Start parents were classified 
as "depressed" at the time of the pretest), although similar to those reported with other 
low-income populations. It is possible that these psychological problems are difficult to 
ameliorate with programs such as Even Start, or at least difficult to change in the short- 
term. Perhaps with continued gains in other areas such as educational attainment and 
employment, parents will express different opinions about their sense of mastery and 
happiness. It also is possible that projects have not focused on these problems and may 
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Exhibit 10.9 








Parent's Expectations for their Child: 
Results from the NEIS 






Parent's 
Expectations 


Number 

of 
Adults 


Average 
Pretest 


Standard 
Deviation 


Gain 


Effect 
Size 


How child will do in 
school" 


3,884 


4.2 


.77 


. 1 O 


on 

.20 


Likelihood of child 
graduating from high 
school b 


4,219 


3.7 


.54 


.09 


.10 


Exhibit reads: Parents are pretest rated the likelihood that their child will graduate from high 
school as 3.7 out of a possible 4. 


•Item rated on five-point scale where "5" equals "very well" and "1" indicates "very poorly". 
"Item rated on four-point scale where "4" indicates "very likely" and "1" indicates "probably will 
not graduate from high school. 
• p < .05. 



be able to help parents through increased collaboration with medical and mental health 
agencies. 

Even Start appears to be having a significant impact in only one area of the home 
environment: the number of reading materials in the home. While all variables on the 
NEIS show a significant change from pretest to posttest but small effect sizes, this is the 
one variable where there also is a significant positive effect seen in the In-Depth Study. 
Since this is one measure out of many, this finding should be interpreted with caution. 
However, many Even Start projects make a concerted effort to increase the number of 
books, magazines and newspapers in participants' homes, either through loans or free 
donations of reading materials. 

There are no significant program effects on the Parent-Child Reading Task, which was 
developed as a new measure to extend the information collected from the In-Depth Study 
beyond the traditional paper-and-pencil tests. In addition, Even Start project directors 
urged us to collect some direct assessment of parent-child interaction and a shared 
literacy activity. However, the negative changes recorded from pretest to second 
posttest on this measure raise questions about the utility of the rating scale in its present 
format. 
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One shortcoming of the Parent-Child Reading Task may. be that it is not sensitive enough 
to differences between parents to show changes over time. The choice to make the 
coding a simple yes/no check of behaviors rather than a count of the number of 
occurrences of a behavior means that we cannot differentiate parents who display 
multiple examples of positive behaviors from those who exhibit the behavior only once. 
Much of the prior research in this area used videotapes and coded mother-child behaviors 
from the tapes. We felt that this would be a costly approach and also would make some 
parents and children feel uncomfortable. 

Another difference between the current measure and the research tools on which it was 
based is the purpose of the observation. Most of the prior research studies (e.g., Lancy 
and Draper, 1988; Pellegrini et al., 1990; Resnick et a!., 1987) were attempting to 
describe the content and quality of parent-child reading in order to determine whether 
mothers incorporated teaching strategies when they read to their child or to relate 
parents' reading strategies to the acquisition of the child's reading abilitv. Thus, using 
this type of instrument to assess program effects is a new application that may require 
more differentiated coding schemes. 
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Chapter Eleven 

Effects of Even Start Projects on Families 



This section presents results from the In-Depth Study on the social supports and financial 
resources available to Even Start families. Information on employment status from the 
NEIS and In-Depth Study also is reported. By providing support services to participating 
families and engaging parents in program activities, it is hypothesized that Even Start may 
help families develop a wider social network and greater access to social services. In 
addition, adult education services could enable parents to find work or change to higher 
paying jobs, with the long-term effect of increasing income from wages and reducing 
reliance on public assistance. In addition, this section describes results from focus groups 
with Even Start parents about their perceptions of the program's impact on their families. 
Subsections include: 

• Social Support for Families 

• Family Resources 

• Employment Status 

• Parents' Perception of Program Impacts 

• Conclusions about Effects of Even Start on Families 



Social Support for Families 

The term "social support" refers to the help and support offered to individuals and families 
by their relatives, friends and neighbors. The availability of this support can affect one's 
psychological well-being by providing direct assistance as well as serving as an informal 
referral source to community services (Gottlieb, 1976). Adequate social support also has 
been linked to outcomes for children. The availability of a support network for parents 
influences children both in terms of the amount of emotional energy that parents have foi 
their children as well as the increased opportunity for interactions and experiences outside 
of the home that provide sources of cognitive and social stimulation for children (Cochran 
and Brassard, 1979). 



Description of the Measure 

The Inventory of Socially Supportive Behaviors (ISSB), developed by Barrera, was included 
in the parent interview for the In-Depth Study to obtain information about parents' social 
supports. This measure was chosen over other measures of social support because it 
includes concrete behaviors in addition to emotional support, uses clear and simple 
language, and assesses the frequency of support rather than perceived satisfaction with 
available supports. 
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The ISSB was designed to assess various types of assistance that people have available 
in everyday life. The scale includes 40 behaviors such as: provided you with a place 
where you could get away for a while, provided you with transportation, told you who 
you should see for assistance, and loaned you over $25. Respondents are asked to rate 
the frequency of each event in the past month on a five-point scale from "not at all" (1 ) 
to "almost every day" (5). The measure is scored by creating either a total additive score 
across all items or an average score if ratings are missing from some items. 

In previous research, the measure has demonstrated adequate psychometric properties. 
The test-retest reliability for the total score was .88 when the measure was administered 
twice over a two-day period (Barrera, 1981). The internal consistency of the total score 
has generally been above .90 (Barrera, no date). For the In-Depth Study, the ISSB scale 
was shortened to 27 items to reduce respondent burden. The internal consistency, as 
measured by Cronbach's alpha, was .92 for the total score based on these 27 items. 



Effects on Social Support 

There is no program effect on the social support scale in the In-Depth Study. This scale 
was not part of the NEIS. Exhibit 11.1 presents the pretest and second posttest scores 
on the ISSB. Parents in Even Start and the control group gave very similar ratings to the 
frequency of social support, and there are no statistically significant differences between 
the groups. The average scores from both the Even Start and control groups are 
somewhat lower than those reported in the research literature (Barrera, no date) in both 
pregnant teens (mean: 2.6) and female college students (mean: 2.5). 



Family Resources 

A number of researchers have discussed the impact of limited financial resources on 
family dynamics, parenting, and child growth and development. Families living in poverty 
are more likely to experience greater and more chronic stress than middle or upper income 
families; in addition, the incidence of major stresses, such as inadequate housing and lack 
of money for food, creates psychological stress that affects parent-child and parent-parent 
interactions within the family (Parker et al., 1988). As a result, poor families are more 
likely to exhibit punitive parenting styles than more economically advantaged families 
(McLoyd, 1990). 



Description of the Measures 

Information about family resources comes from three areas of the parent interview: a 
question about the major sources of household income, a question categorizing the level 
of family income, and a set of questions about the adequacy of family resources. 

The family resource scale is an adaptation of an instrument developed by Leet and Dunst 
for use in family intervention programs (Dunst et al., 1988). It is one of the few 
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Exhibit 11.1 

Social Support: Results from the In-Depth Study 




Even Start 
(n = 81) 


Control 
(n=73) 


Mean 


(s.d) 


■i 

Mean 


| (s.d) 


Inventory of Socially Supportive 
Behavior" 

Pretest 2.4 
Second Posttest 2.3 
Gain -0.05 

Program Effect 
(Effect size) 


(0.9) 2.2 (0.8) 
(0.7) 2.3 (0.8) 
0.03 

-0.08 points 
(.10 s.d.) 


Exhibit reads: Even Start parents scored an average of 2.4 out of 5.0 at pretest on the 1SSB scale. 
'Score represents average across 27 items rated on five-point scale, where "5" indicates greater 
frequency of social support. 



measures we have seen that tries to assess the adequacy of resources for basic 
necessities (e.g., food for two meals a day, indoor plumbing, and owning or having access 
to a telephone). The scale includes 25 items which respondents are asked to rate on a 
five-point scale from "not at all adequate" (1) to "almost always adequate" (5). An 
average rating across the 25 items was created as a composite score. 

In prior research, the measure has shown adequate psychometric properties. The authors 
report a Cronbach's alpha of .92, a split-half reliability of .95 corrected for length using 
the Spearman-Brown formula, and a test-retest reliability of .52, based on administering 
the scale to 45 individuals on two occasions two months apart (Dunst and Leet, 1987). 
In the In-Depth Study, the internal consistency of the composite score was .85. 



Effects on Family Resources 

There is a statistically significant increase in the adequacy of family resources reported 
by Even Start families; however, control group families also gained a small amount and 
there is no significant program effect. There are no program effects on income or reliance 
on government assistance. The lack of effects on income is to be expected, since 
changing income is a relatively long-term outcome for Even Start. 

Exhibit 1 1 .2 presents the results on the Family Resource Scale for Even Start and the 
control group. Both groups reported an average of 3.6 out of a possible 5.0 on the 
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Exhibit 11.2 








Adequacy of Family Resources: Results from the In-Depth Study 






Even Start 
<n = 81) 


Control 
<n=73) 




Mean 


(s.d) 


Mean 


(s.d) 


Family Resource Scale" 










Pretest 

Second Posttest 
Gain 


3.6 

r.7 

0.12* 


(0.6) 
(0.6) 


3.6 
3.7 
0.04 


(0.6) 
(0.5) 


Program Effect 
(Effect size) 




0.08 points 
(.13 s.d.) 




Exhibit reads: Even Start families scored an average of 3.6 out of 5.0 at pretest on the Family 
Resources Scale. 

'Score represents average across 25 items rated on five-point scale, where "5" indicates more adequate 
resources. 


*p<.05. 











pretest, indicating that they perceived their resources to be between "sometimes 
adequate" and "usually adequate" at the time they entered Even Start. This suggests that 
parents did not perceive a large problem with a lack of resources. In both groups, the 
average rating of the adequacy of family resources rose slightly from pretest to second 
posttest. This gain was statistically significant for the Even Start participants; however, 
the difference was not large enough when compared to the gain among the control group 
to show a statistically significant program effect. 

Exhibit 1 1 .3 shows the percentage of families at pretest and posttest whose household 
income was less than $10,000, and who receive government assistance. While the 
percentage of Even Start families with income below $10,000 dropped from 72 percent 
at pretest to 65 percent at the second posttest, control group families showed a similar 
decrease. 

A second way to analyze these data is to focus on families whose income was below 
$10,000 at the pretest to see how many were able to increase their income above this 
level. Exhibit 11.4 shows that 20 percent of Even Start families and 21 percent of 
control families reported in., eases that moved their income from under to over $10,000. 
The difference between groups is not statistically significant. 
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The percentage of Even Start families on government assistance remained fairly constant 
from pretest (48 percent) to second posttest (47 percent), while the percentages among 
the control group dropped slightly from 45 percent at pretest to 41 percent at the second 
posttest (see Exhibit 11.3). These differences are not statistically significant. Exhibit 
11.5 shows the percentage of families in each group who were able to get off of public 
assistance in Even Start (26 percent) and the control group (22 percent). Once again, 
these differences are not statistically significant. 



Employment Status 

Although Even Start is not primarily an employment and training program, the adult 
education component of the program could effect changes in parents' employment status 
by increasing their skill levels and employability. 



Description of the Measure 

The parent interview for the NEIS and the In-Depth Study asked respondents to indicate 
their current employment status as working full-time, working part-time, or not working. 
For the NEIS, these questions were asked at program entry and again either at the end 
of the program year or at exit. In the In-Depth Study, employment status was asked at 
the pretest, first posttest and second posttest. The information was specific to the adult 
targeted for the evaluation and does not necessarily reflect whether anyone in the 
household was employed. 

The analyses focused on the percentage of adults who were not employed versus those 
working part-time or full-time. We aggregated data about part-time and full-time 
employment for a number of reasons. First, Even Start project directors pointed out that 
for mothers of young children, part-time work is a positive outcome. Also, part-time 
employment in a job with higher wages can be an improvement over a full-time but low- 
paying job. 
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Exhibit 11.3 




Financial Resources: Results from the In-Depth Study 




Even Start 


Control 




(n = 84) 


(n = 75) 


Household Income Less than $10,000 






Pretest 


72.3% 


72.6% 


Second Posttest 


65.1% 


64.4% 


Gain 


7.2% 


8.2% 


Program Effect (Effect Size) 


-1.0% (.01 s.d.) 


Receiving Government Assistance 






Pretest 


48.2% 


45.1% 


Second Posttest 


46.9% 


40.9% 


Gain 


1.3% 


4.2% 


Program Effect (Effect Size) 


-2.9% (.06 s.d.) 


Exhibit reads: 72 percent of Even Start parents had household incomes less than $10,000 at pretest. 
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Exhibit 1 1 .4: Percentage of Families Who Increased Income 
Above $1 0,000 by Posttest In-Depth Study 



, 20%y I 











)21%j 


; m :;;;; : i:f;j20%i • 









































1- 



Ev»n Start 



Control 



Exhibit reads: Of those families whose household income was less than $ 1 0,000 at pretest, 20 percent of Even 
Start families and 21 percent of control families reported incomes above $10,000 at the second posttest. 
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Exhibit 1 1 .5: Percentage of Families Who Got Off Public 
Assistance as Primary Source of Income 
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Exhibit reads: Of those families whose primary source of income was public assistance, 24 percent of Even 
Start families and 16 percent of control families no longer relied solely on public assistance by posttest. 



Effects on Employment Status 

Based on results from the In-Depth Study and the NEIS, Even Start does not appear to 
have a significant effect on participants' employment status. 

Effects as Measured by the In-Depth Study. Exhibit 1 1 .6 shows the percentage of 
parents in the Even Start and control groups who were employed at the pretest and 
second posttest. While some adults in each group who were not employed at pretest 
became employed by the second posttest, there is no significant program effect. When 
the sample is restricted to just those adults who were not employed at pretest, fewer 
adults in Even Start (14 percent) than the control group (22 percent) fcund work by the 
second posttest (Exhibit 11.7). This may be due to the greater likelihood that Even Start 
adults are involved in adult education programs or other training and are not yet seeking 
employment. Once again, this difference is not statistically significant. 

Effects as Measured by the NEIS. Exhibit 1 1 .8 presents information about the percentage 
of adults who were not employed at program entry for the total group of respondents and 
for subgroups of participants. Approximately 78 percent of the adults participating in 
Even Start were not employed at pretest. Nearly 1 3 percent of these participants found 
employment by the end of the program year or by the time that they exited from Even 
Start. The results are fairly consistent across subgroups of participants. 
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Exhibit 11.6 






Employment Status: Results from the In-Depth Study 




Even Start 


Control 




(n = 84) 


(n = 73) J 


Employed Part-Time or Full-Time 






Pretest 


17.6% 


20.5% 


Second Posttest 


22.4% 


30.1% 


Gain 


4.8% 


9.6% 


Program Effect 


-4.8% 


Exhibit reads: 17.6 percent of Even Start parents were employed either part-time or full-time at 


pretest. 











Exhibit 1 1 .7: Percentage of Adults Not Employed at 
Pretest who Found Employment by Posttest 
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Exhibit reads: Of those adults who were not employed at pretest, 12 percent of adults in Even Start and 15 
percent of adults in the control group were employed at the second posttest. 
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Exhibit 11.8 








Employment Status: Results from the NEIS 








All Participants 


Not Employed at Intake 


Group 


Total 


Not Employed at 
Intake 


Total 


Found 
Employment 


Adults 


N 


% 


Adults 


N 


% 


A rio at Intflif o 
nyc at IlllatvC 














1 InHpr 9 1 


1,764 


1 ,000 


Q Q O 
OO.O 


1,453 


1 EL A 

1 b4 


10. D 


91 9R 


4,417 


o,ob I 


oz.y 


3,430 


A 1C 

436 


1 2.7 


zu ou 


4,163 


*3 1 QA 

O, I iJ L T 


7fi 7 


3,019 


/HQ 
4 i y 


i o.y 


31-35 


2,328 


I ,DDU 


71 *3 
/ I .O 


1,555 


zi y 


*i A 1 

14. 1 




1,137 


QIO 

o I z 


"71 A 

l\ .4 


765 


97 


1 2.7 


Over 40 


789 


R77 


7*3 1 
/O.I 


537 


CO 

DZ 


Q "7 

y . / 


Gender 














Male 


1,284 




AR Q 


544 


1 1C 

1 1 O 


zl . 1 


Female 


13,369 


in SfiQ 

1 KJ, OU3 


R 1 "3 
O I .O 


10,220 


1 ORE 


1 z.4 


English is primary language 












No 


4,018 


O 1 IE 
O, 1 1 0 


/ / .D 


2,884 


oao 


1 o o 
1 o.o 


Yes 


10,874 


8,566 


TO O 

/o.o 


8,073 


QQA 

yyu 


1 z.o 


Ethnic background 














Asian 


482 


360 


74. 7 
/ f. / 


312 


ZD 


o.U 


African American 


4,805 


3,804 


7Q 0 

1X3./. 


3,496 


o /y 


1 U.o 


Hispanic 


3,783 


2,948 


77.9 


2,799 


391 


14.0 


Native American 


795 


672 


84.5 


648 


93 


14.4 


White 


5,038 


3,883 


77.1 


3,726 


508 


13.6 


Attained GED while in Even Start 










No 


1 1 ,094 


8,997 


81.1 


8,384 


979 


11.7 


Yes 


1,006 


809 


80.4 


793 


144 


18.2 


TOTAL 


15,191 


11,910 


78.4 


11.169 


1,414 


12.7 


Exhibit reads: At intake, more than three quarters (78.4 percent) of Even Start adults targeted for the 
evaluation were not employed. Of those who were not employed, 12.7 percent found part-tima or full- 
time employment while participating in Even Start. 
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Parents 9 Perceptions of Program Impacts 



As part of the In-Depth Study, parents in Even Start were asked for their own assessment 
of the program's impact. 

Description of the Measure 

During annual site visits to the ten In-Depth Study projects, Abt staff conducted focus 
groups with Even Start parents to obtain their views of the program. These discussions 
were held with approximately ten parents at each site. Project directors invited parents 
to take part in the focus group, but their participation was voluntary. Topics for the focus 
groups included the reasons why parents joined Even Start and the impact of the program 
on themselves and their family. 



Perceived Effects on Adults 

Parents described a number of positive effects of their participation in Even Start, 
including improved personal skills, increased parenting ability and learning gains. In some 
programs parents noted that the main reason they joined Even Start was for the adult 
education. Some parents were interested in preparing for the GED certificate, continuing 
their education or improving English proficiency. Others stated that education was a way 
to improve their lives. As one woman said, "I want to better myself, to take up a trade, 
to be a nurse's aide." Another parent indicated a desire to "do something with my life." 

Participants commented that Even Start recognizes that the adult learners also are 
parents. One mother stated that she had once considered attending classes to prepare 
for the GED but because the program did not offer child care, she was not able to attend. 
Several women had dropped out of night school prior to joining Even Start; one of these 
woman indicated that she liked Even Start "because it was different from other school" 
experiences she had had either as a child or an adult. Several parents commented that 
having the early childhood services available was a plus for them as well as their children. 
For some it is the convenience, for others it is the safety-knowing where their children 
are and being able to check on them during the day eased parents' concerns. 

Even Start participants also discussed the changes they experienced in their interactions 
with their children. Although most parents indicated that the parenting classes were not 
the main reason for joining the program, they now viewed this component as extremely 
important. Through the parenting classes and home visits, parents described being able 
to communicate more effectively with their children. The program also has helped them 
learn how to play with their children--to be creative with their children, to make games 
for their children, and to plan activities for their children at home. 

Parents described being able to interact more positively with their children and deal with 
the stresses they feel as a parent without "exploding." Several parents spoke of having 
more patience with their children and using less physical punishment. One mother told 
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how she has learned not to criticize her children as much, but rather to give them praise; 
further, she felt that she has developed more realistic expectations about her children and 
herself. 

A number of parents commented that they have higher personal goals and more self- 
esteem as a result of Even Start. One woman stated that before she joined the program, 
she viewed the GED as "working papers," but now she sees the certificate as "the first 
step towards a better life." 

The program also has reduced the parents' sense of isolation and being "trapped at home 
with their children." Although some of the women knew each other before enrolling in 
the program, they had only limited interaction. As one mother said, "you can live right 
next door to someone and not know them." Now they check up on each other and visit 
or call if someone has missed class. Another mother commented that "for most of us, 
it's just us and our kids." Even Start has given them the opportunity to interact with 
other adults who have similar problems and experiences. As a result, participants feel 
better able to cope with life situations. 

In several programs, participants described Even Start as their "family." In one project, 
parents gave staff high marks for being respectful and willing to do anything for the 
familiss-"we just pick up the phone and tell them we need help and they are there." In 
another site, parents told how staff treat them "as human beings without putting you 
down or judging you," but all the while "helping us in a new way of life." In a third site, 
parents pointed out that the friendly and supportive environment of the program allows 
parents to feel valued and know that their concerns are taken seriously. 



Perceived Effects on Children 

Parents were quite positive about Even Start's effects on their children. They reported 
that their children's attitudes toward reading and schooling had changed to excitement 
and eagerness to learn. Several parents cited specific skills that their children have 
learned, such as using scissors, learning the alphabet and the name of colors. Parents 
with older children stated that their younger children in Even Start are much better 
prepared for public school than their older siblings-they know how to take the bus, they 
can read, they speak English better and they are "just less afraid." 

In addition to cognitive skills, improved verbal and social skills were reported by many 
parents. One mother stated that her child rarely spoke before he attended Even Start, and 
"now he talks constantly." Others commented that the program has helped their children 
to be less shy, to speak more clearly, and to separate more easily from their mothers. As 
one mother described it, " my child does more for himself in everything-in eating, in 
dressing himself. ..he isn't always hanging onto me anymore." 

Parents reported that their children are now better behaved. One mother described her 
son's pre-Even Start activity level as similar to "a grasshopper," but now he is more calm 
and controlled. Parents also described their children as more helpful and cooperative at 
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home. The organizational skills that children learn at school (e.g., putting things back in 
their place) have carried over to the home as well. 



Conclusions about Effects of Even Start on Families 

The In-Depth Study included several measures of the effects of Even Start on participating 
families, including their perceived social support, adequacy of financial resources, income 
level and sources, and employment status. The NEIS instruments had questions about 
income and employment. Across these measures, gains from program entry to the end 
of the second program year are minimal. It is possible that these areas are difficult to 
change because they are affected by circumstances beyond the control of the program, 
such as the local economy and the availability of jobs. It is also likely that these indices 
will not show large changes in the short-run, but instead require longer interventions and 
other positive short-term impacts in order to achieve significant gains. 

While there are few effects for families on these quantifiable variables, Even Start 
participants report quite a number of qualitative changes in their lives and the lives of their 
children. Listening to the personal stories cf program participants, it becomes apparent 
that most of the changes in attitudes and skills that the parents see in themselves and 
their children are positive short-term impacts of the program. Parents describe 
themselves as moving toward their goals of an educational certificate, getting a job and 
being a better parent. Given the current status of the Even Start parents, it is reasonable 
that we do not yet see changes in the more distal outcomes of increased employment and 
income. 
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Chapter Twelve 

Cost of Even Start 



Even Start's budget has grown steadily since its inception, from $14.5 million in 1989 
to an estimated $90 million in 1993. As Exhibit 1 .1 showed, this increase has enabled 
Even Start to fund more projects each year. Data on federal costs and number of families 
served are available for all Even Start projects, while projects in the In-Depth Study 
portion of the evaluation participated in a special assessment of the federal and local 
costs of Even Start. While the In-Depth Study projects were not randomly selected, but 
they do represent a reasonable cross-section of fully functioning Even Start grantees. 
This section of the report includes discussions of: 

• Cost per Participating Family 

• Distribution of Total Cost by Function 

• Project Variation in Federal Cost per Family 



Cost Per Participating Family 

Even Start projects engage in many functions including project administration and 
coordination, three core services, a range of support services, evaluation activities, 
recruiting, case management, and many others. One aim of this evaluation was to 
calculate the federal (Even Start) cost per family; a second purpose was to ascertain the 
full cost of all Even Start services, including federal Even Start funding, local matching 
funds, in-kind services or facilities, the value of other locally obtained core and support 
services, and other federal funding (e.g., the pro-rated cost of early childhood education 
obtained from Head Start or the pro-rated cost of adult basic education obtained from a 
local community college). 

Exhibit 12.1 shows the federal cost for Even Start projects participating in the national 
evaluation for the first three years of program operations. The number of families and 
participants (adults and children) served increased greatly from 1989-90 to 1990-91, as 
would be expected given the increase both in number of projects funded and total federal 
dollars spent on the program. The number of projects in the evaluation remained constant 
from 1990-91 to 1991-92; however, the number of families and participants increased 
substantially once again (by 46 percent), indicating that projects were maturing and 
becoming more efficient. 
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Exhibit 12.1 






Federal Cost for Even Start Projects, 
by Program Year 




Measure 


1989-90 


1990-91 


1991-92 


Number of Projects 


76 


123 


123 


Total Grant Awards (millions) 


$14.5 


$24.2 


$24.2 


Average Grant Award 


$190,789 


$196,748 


$196,748 


Total Families Served 


2,460 


6.596 


9,668 


Average Number of Families 


32 


54 


79 


Total Participants Served 


5,672 


16,143 


OA 7QQ 
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Average Participants Served 


75 


131 


202 


Federal (Even Start) Cost Per 
Family 


$5,894 


$3,669 


$2,503 


Federal (Even Start) Cost Per 
Participant 


$2,556 


$1,499 


$975 


Exhibit reads: The federal (Even Start* cost for each Even Start family was $2,503 during the 1991- 
92 program year. 



The federal cost per Even Start family declined over time, from $5,894 in 1989-90 to 
$3,669 in 1990-91, and again to $2,503 in 1991-92 11 . The same pattern holds for the 
cost per participant-a decline from $2,556 in 1989-90 to $975 in 1991-92. 

Data from projects participating in the In-Depth Study cost analysis show that Even Start 
projects obtain substantial amounts of support, in addition to their federal Even Start 
funds, in order to deliver appropriate services to participating families. As noted above, 
the federal (Even Start) per family cost was $2,503 in 1 991-92. The In-Depth Study cost 



11 This calculation is based on total federal costs divided by the total number of 
families served and is appropriate for costing policy alternatives and making comparisons 
among federal programs. It also is possible to calculate the federal cost per family for a 
typical Even Start project-the mean cost per family is calculated for each project in the 
study and then an average of project means is computed. This approach gives equal 
weight to each project (rather than to each family), and yields a cost per family of $3,634 
in 1991-92. The project-based cost per family is higher than the individual-based cost per 
family because large projects (which serve more families at a lower per-f amily cost) count 
no more than small projects (which serve fewer families at a higher per-f amily cost). This 
approach would be appropriate if the interest is in comparing costs across projects, or in 
determining which types of projects are most cost-efficient. 
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analysis found that local projects obtained additional funding of $1,352 per family, for a 
total of $3,855 per family (see Exhibit 12.2). 



Exhibit 12.2: Federal Even Start Cost and Ail Other Costs 
(1991-92 Program Year) 



Federal Even 
Start Costs 
(65 percent) 




All Other Costs 
(35 percent) 



Total Coats - $3,866 per famUy; 

-$1,602 per participant 



Exhibit reads: Sixty-five percent of total Even Start costs were provided by federal Even Start funding; 35 
percent were provided by other sources including local matching, in-kind services or facilities, the value of other 
locally obtained core and support services, and other federal funding (e.g., the pro-rated cost of early childhood 
education obtained from Head Start). 



Thus, federal Even Start funds comprise 65 percent ($2,503) of the total cost per family 
($3,855), and other funds obtained locally (including local, state, and other federal) 
comprise 35 percent ($1,352) of the total cost per family. Alternatively, local projects 
are able to match 54 percent of their federal Even Start funds. 



Distribution of Total Cost by Function 

More than half (55 percent) of all Even Start costs were incurred in the provision of core 
services: 31 percent for early childhood education, 15 percent for adult education, and 
9 percent for parenting education (Exhibit 12.3). An additional 9 percent was spent on 
the provision of support services which are designed to enable families to participate in 
core service activities. Thus, almost two-thirds (64 percent) of projects' funds were 
spent on the direct provision of services. Remaining funds were spent for program 
administration and coordination (14 percent), evaluation (10 percent), case management 
and recruiting (4 percent), and for a variety of other functions (8 percent) such as field 
trips, staff meetings, clean-up, and errands. 
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Exhibit 12.3: Distribution of Federal Even Start Dollars 
by Function (1991-92 Program Year) 

Early Childhood Education 



Parenting / 
Education / 



Adult Education 




Administration 



Other 



Case Mgi & 
Recruiting 



Support Svcs. 



Evaluation 



Exhibit reads: 31 percent of Even Start costs (both federal and ail other) were incurred in the provision of early 
childhood education services. 



Project Variation in Federal Cost Per Family 

As with most variables investigated in this evaluation, there is great variation across 
projects in the amount of federal dollars spent on each family. Exhibit 12.4 shows a 
distribution of project-level per family costs. 12 Some projects spend relatively few 
federal dollars per family while other projects spent much more on a per family basis. 
Most projects spent between $1,000 and $5,000 federal dollars per family. However, 
six projects spent less than $1,000 per family, while three projects spent over $8,000 
per family. Such wide variation in expenditures per family reaffirms that Even Start 
projects took very different approaches to organizing and implementing services. It also 
suggests that there are wide project-to-project differences in access to federal and non- 
federal resources outside of Even Start. For example, some projects used locally-available 
adult education and early childhood education services, while others had to provide those 
services using their federal Even Start funds. 

Information on two project characteristics that are related to cost per family is shown in 
Exhibit 1 2.5. One variable that is related to federal cost per family is the locus of primary 
responsibility for providing adult education and early childhood education services. As 



12 The federal cost per family at the project level is calculated by dividing the total 
number of families participating in core services during a year by the total federal funds 
received by the project for that year. 
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Exhibit 1 2.4: Project-Level Distribution of 
Federal Cost Per Family (1991-92 Program Year) 
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Exhibit reads: Six projects spent less than $1 ,000 of their federal grant per family, while three projects spent 
over $8,000 per family. 



would be expected, projects that provided their own services expended more of their 
federal dollars on each family than projects that delegated responsibility for providing 
services to a cooperating agency. Eight projects retained primary responsibility for 
providing adult education and early childhood education. These projects spent a relatively 
large amount ($5,775) of their federal grant on each family because they paid for the 
direct provision of services. Eighty-three projects shared responsibility for providing core 
services with cooperating agencies. On average, these projects spent $3,579 of their 
federal dollars on each family. Finally, eight projects delegated primary responsibility for 
providing adult education and early childhood education to cooperating agencies. These 
projects spent an average of $1,878 of their federal dollars on each family. 

A second variable that relates to the amount of a project's federal grant spent on each 
family is the number of families served by the project. As would be expected, projects 
that served large numbers of families spent fewer federal dollars per family, while projects 
that served smaller numbers of families spent more federal dollars per family. 

In 1 991 -92, there were 1 8 projects that served fewer than 30 families. On average, they 
spent $6,31 2 per family. The 66 projects that served between 30 and 99 families spent 
$3,742 per family. And finally, the 28 largest projects, which served 100 or more 
families, did so at a federal cost of $1,659 per family. 
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Exhibit 12.S 




Federal Cost Per Family by Project Size and 
Locus of Responsibility for Core Services 
(1991-92 Program Year) 


Project Characteristic 


Number of 
projects 


Average Grant 
Per Participating 
Family 


Locus of primary responsibility for adult education and early childhood education 
services 


Even Start 


8 


$5,775 


Both 


83 


$3,579 


Cooperating agency 


8 


$1,878 


Number of participating families 






Less than 30 families 


18 


$6,312 


30-99 families 


66 


$3,742 


100 or more families 


28 


$1,659 


Exhibit reads: Projects responsible for directly providing core services had higher costs than projects 
which relied on cooperating agencies. Small projects had higher costs than large projects. 


Note: Only projects reporting at least ten participating families were included in the analysis. 
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Highlights 

Authorized by Congress in 1988, the Even Start Family Literacy Program is designed to 
improve the educational opportunities of the nation's children and adults by providing 
three core services to participating families: early childhood education, adult education, 
and parenting education. Federal Even Start funds are used for coordinating existing 
services as well as for the direct provision of services which are not locally available. 



Implementation of Even Start 

• During its first four years Even Start provided for the implementation 
of several hundred projects which delivered family literacy services 
to more than 20,000 families at a Federal cost of about $2,500 per 
family per year, [see Chapter 4, p. 41 and Chapter 12, p. 231 -236] 

• Even Start served its intended population. Of all Even Start adults 
served, 79 percent did not complete high school, and 66 percent of 
Even Start families had total annual income under $10,000. The 
average adult entered Even Start with the literacy skills of a high 
school student, while the average three- or four-year-old child who 
entered Even Start scored at the 9th percentile on a nationally- 
normed vocabulary test, [see Chapter 4, p. 39-70] 

• The average Even Start family participated in the program for seven 
months and received an average of 13.5 hours of adult education, 
6.5 hours of parenting education, and 2 hours of early childhood 
education each month, [see Chapter 6, p. 137-145] 

• Virtually all Even Start projects offered the three required core 
instructional services of adult education, parenting education, and 
early childhood education. They also offered some instructional 
services in a home-based setting, some services to parents and 
children together, and appropriate support services to enable families 
to participate fully in Even Start's core services, [see Chapter 5, 
p. 72-81] 
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Effect of Even Start on Program Participation 

• Adults and children in Even Start families participated much more 
frequently in educational services than they would have had they 
not been in the program: 91 percent of Even Start families 
participated in adult education vs. an estimated 30-40 percent 
without the program, 95 percent participated in parenting education 
vs. an estimated 8 percent without Even Start, and 98 percent 
participated in early childhood education vs. an estimated 60 
percent in the absence of Even Start, [see Chapter 6, p. 1 27-1 31 ] 



Effect of Even Start on Parents 

• Even Start helped many adults attain a GED. Across all 120 
projects in the evaluation, 8 percent of adults who entered Even 
Start without a GED or diploma achieved one. In a randomized 
experimental study of a subset of 5 Even Start projects, significantly 
more adults in Even Start than in the control group attained a GED 
(22 percent vs. 6 percent), [see Chapter 9, p.192-198] 

• Adults who participated in Even Start achieved significant positive 
gains (about 4-5 points, or 1/3 of a standard deviation) on the 
CASAS reading survey (a measure of adult functional literacy in 
reading), gains which are comparable or greater in size than those 
observed in other studies of adult education programs. However, 
families in a control group achieved similar gains, [see Chapter 9, 
p. 183-192] 

• Even Start did not produce any measurable effects on parenting 
skills as assessed by self-report measures of the home learning 
environment, parenting activities, parental expectations, sense of 
mastery, and depression; and in-home observations of parents 
reading to their children. This could be due to a weakness in our 
ability to measure these areas, to generally positive parent responses 
and high self-ratings at pretest, or to a lack of program 
effectiveness, [see Chapter 10, p. 202-21 8] 

• Even Start did not produce any measurable effect on family 
resources, income, or employment. It is unlikely that effects in 
these areas would be evident in the relatively short time period 
covered by this study, [see Chapter 11, p. 21 9-230] 



Effect of Even Start on Children 

• Even Start children made learning gains, both in terms of school 
readiness skills and on a vocabulary test. 
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By participating in Even Start, children learned school readiness skills 
such as colors, shapes, and sizes (as measured by the PreSchool 
Inventory) significantly faster than would be expected on the basis 
of normal development. Further, after one year of participation, 
Even Start children had significantly higher scores on the PSI than 
children in the control group. However, control group children 
caught up once they entered preschool or kindergarten, [see Chapter 
8, p. 157-165] 

When they entered the program Even Start children scored quite 
low, at the 9th percentile nationally, on the Peabody Picture 
Vocabulary Test (a test of receptive language). During their 
participation in Even Start, children gained a significant amount on 
the PPVT (so that they scored at the 19th percentile nationally). 
However, children in a control group gained a similar amount, [see 
Chapter 8, p. 165-173] 



Relationship Between Amount of Service and Outcomes 

• A high level of exposure to program services is important. Adults 
and children with high levels of participation in Even Start's core 
services gained significantly more on tests than adults and children 
with low levels of participation, [see Chapter 8, p. 180-1 81 and 
Chapter 9, p. 189-192] 

• The extent to which parents took part in parenting education is 
significantly related to gains in children's vocabulary (as measured 
by the PPVT), over and above gains in vocabulary that result from 
children participating in early childhood programs. Families that 
participated in a relatively low-intensity or moderate-intensity 
program had children who scored at the 17th to 19th percentile, 
while families that participated in a relatively high-intensity program 
had children who scored at the 26th percentile [see Chapter 8, p. 
175-180]. 

To sum up, Even Start has had some positive short-term effects on children and adults, 
although those effects vary greatly across projects. In fact, there is great project-to- 
project variation in all facets of Even Start-program planning and implementation, 
characteristics of families served, participation, and outcomes. Even Start's services did 
result in gains for children and their parents, but on average, the gains are not greater 
than those that similarly motivated families would obtain for themselves using locally 
available services. In order for Even Start to have a larger effect, it must provide services 
more intensively. 

Even Start families that were intensively engaged in core services did better than families 
with lower levels of participation. Further, there are indications that providing parenting 
education to parents has positive effects on their childrens' vocabularies. This type of 
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parent-to-child effect is encouraging in that it is exactly what Even Start hopes to 
produce. While other early childhood programs have demonstrated positive short-term 
effects on children, and some adult education programs have demonstrated positive 
effects on adults, Even Start is one of the few programs that has focused on parent 
literacy, parenting skills, and child development all at the same time. 

In the long run, it is hoped that Even Start's effects on children will be enhanced because 
of the positive changes made in their parents. Follow-up studies of Even Start 
participants could be conducted to determine whether this long-term portion of Even 
Start's theory holds-that positive effects on parents will lead to long-term positive effects 
for children's cognitive development, social-emotional development, and behaviors. 

The remainder of this executive summary expands upon these highlights. A full 
evaluation report is available from the U.S. Department of Education, Planning and 
Evaluation Service, or from Abt Associates Inc. of Cambridge, Massachusetts. 



Characteristics of Even Start Participants 

Based on data reported for the 1989-90 through the 1992-93 program years, Even Start 
projects are serving the intended population. All participating Even Start households had 
at least one child between birth and age seven, 79 percent of the adults who participated 
in Even Start core services did not complete high school, and 66 percent of Even Start 
families had total annual income (earned income plus public assistance) under $10,000. 
The Even Start population can be further described as follows: 

• 50 percent of Even Start families describe themselves as couples 
with children, 37 percent are single parent households, and 13 
percent have extended families or other living arrangements. 

• 46 percent of Even Start families report job wages as their primary 
source of financial support, while 49 percent report that government 
assistance is their primary source of support. 

• Most adults in Even Start are ages 22-29 (46 percent) or 30-39 (31 
percent). 

• 40 percent of Even Start adults are white, 26 percent are African 
American, 22 percent are Hispanic, 4 percent are Native American, 
and 8 percent are Asian or Pacific Islander. 

• English is the primary language for 66 percent of Even Start adults; 
Spanish is the primary language for 26 percent. 

• Seven percent of the children served by Even Start were identified 
as having a disability. 
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• The average age of Even Start children dropped from 4.3 years in 
1989-90 to 3.7 years in 1992-93, reflecting a federal emphasis on 
serving younger children. 

• Even Start children scored low, at the 9th percentile nationally, on 
a vocabulary test given when they entered the program. 

• Even Start adults attained high school level scores on a functional 
literacy reading test given when they entered the program. 

LEP Adults. Even Start adults with limited English proficiency (LEP) can be characterized 
as follows: 86 percent were educated outside of the United States, 60 percent did not 
reach the ninth grade, 78 percent were not employed at the time they joined Even Start, 
83 percent had an annual income of less than $15,000, and 18 percent were single 
parents. 

Adults Who Enter with a Diploma or GED. Twenty-one percent of the adults who 
participated in Even Start entered already having attained a high school diploma or a GED. 
While this is less than the 33 percent of adults that enter regular adult education 
programs with a diploma, questions have been raised about the fairness of serving these 
potentially less-needy adults in Even Start. Data show that adults who enter Even Start 
with a diploma or GED have characteristics that still suggest the need for Even Start 
services: 40 percent were single parents, 67 percent were not employed, 78 percent had 
annual income under $15,000, and 54 percent relied on government assistance for their 
primary source of income. In addition, the average CASAS reading score for these adults 
is 233, quite close to the average of 230 for adults who reached grades 9-12 prior to 
joining Even Start. This shows that, in spite of their credentials, adults who entered the 
program with a GED or diploma are not functioning at a higher level than their less- 
credentialed counterparts. 



Implementation of Even Start Projects 

Virtually all Even Start projects offer the three required core instructional services of adult 
education, parenting education, and early childhood education. They also offer some 
instructional services in a home-based setting, some services to parents and children 
together, and appropriate support services to enable families to participate fully in Even 
Start's core services. 

Early Childhood Education. Children in Even Start projects were provided a range of early 
childhood education services: 

• 67 percent of the projects enrolled some of their children in Head 
Start, 50 percent enrolled some of their children in a Chapter 1 pre-K 
program, and 87 percent provided some other preschool option. 
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• For children old enough to enter the public schools, most Even Start 
projects participated in joint planning activities with the public 
schools. Hence, 78 percent of the projects included kindergarten as 
an Even Start service, and 70 percent provided early childhood 
education services to children under age eight who were in primary 
grades, again through joint planning with the public schools. 

Adult Education. Almost all (93 percent) of the projects reported that they provided 
services to prepare adults to attain a GED certificate, 85 percent provided services in 
adult education, 81 percent provided services in adult secondary education, and 61 
percent provided instruction in English as a second language (ESL). 

Parenting Education. Even Start projects provided a wide range of services to help 
parents understand and enrich their child's development. For example, projects helped 
families make use of services provided by other social service agencies, discussed 
parents' role in the education of their children, oriented parents and children to school 
routines, taught parents about child development, trained parents in child behavior 
management, worked to build parents' self-esteem, and instructed parents in life skills and 
in principles of health and nutrition. Each of these different types of parenting education 
was provided by 90 percent or more of the Even Start projects. 

Adult/Child Services Delivered Through Even Start. More than 90 percent of the Even 
Start projects reported that they delivered core services to parents and children together. 
This is an important part of the Even Start model in that it impresses on parents that they 
are key to their child's education. Examples of services delivered to admits and children 
together include reading and storytelling, developing readiness skills, social development 
and play, development of gross motor skills, working with numbers, arts/crafts, and 
health/nutrition. 

Support Services. Support services remove barriers that could restrict a family's ability 
to participate in Even Start core educational services. Several types of support services 
were provided by 80 percent or more of the projects including transportation, family 
advocacy assistance, nutrition services, counseling services, and child care. 

Cooperative Arrangements. Even Start projects are required to establish cooperative 
arrangements with other agencies to avoid duplicating services. This strategy allows 
optimal use of limited resources so that projects can concentrate on filling service gaps. 
Collaboration and cooperative arrangements were, indeed, a key focus of Even Start 
projects. During the 1992-93 program year, Even Start projects were involved in more 
than 6,000 cooperative arrangements to provide core services, an average of 20 
cooperative arrangements per project. Forty-two percent of the arrangements were for 
parenting education, 27 percent were for adult basic education, and 31 percent were for 
early childhood education. The most common cooperators were "other departments and 
programs within the public schools," "local, county, state or tribal agencies," and 
"postsecondary institutions." 

Implementation Problems. Even Start projects reported several barriers to implementation. 
The most common barriers were difficulties in the recruitment, retention, attendance, and 
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motivation of families, problems of communication and coordination with cooperating 
agencies, lack of transportation for families, the unexpectedly severe social service needs 
of the families and community, lack of quality child care, financial problems, staffing 
problems, and problems with facilities and space. 

Staffing Even Start Projects. Most Even Start projects have a project administrator and 
three to five staff who provide instruction in early childhood education and adult 
education. About half of the project administrators have multiple job responsibilities, 
including coordination or supervision in each of the three core services, recruiting families, 
and project evaluation. 

• Approximately 54 percent of the staff providing services to Even 
Start families are paid by collaborating agencies, 32 percent are paid 
through the federal grant, and 13 percent are paid with local 
matching funds. Staff from collaborating agencies are most likely to 
provide direct instruction in early childhood education and adult 
education, although their classrooms may have a small number of 
Even Start participants. 

• Even Start project administrators have an average of nine years of 
work experience in early childhood education programs, four years 
in adult education programs, and six years with parenting education. 

• Early childhood education teachers have an average of eight years 
of work experience in that field, while adult education teachers have 
an average of five years of work experience in adult education 
programs. 

One-third of Even Start staff have a college degree, 20 percent more have a master's 
degree, and an additional 9 percent have a doctorate. Those with a master's degree or 
beyond tend to be project directors, coordinators, or staff involved in the evaluation. 
More adult education teachers (36 percent) have a master's or doctorate than early 
childhood teachers (20 percent) or parenting education teachers (28 percent) . Nearly one- 
quarter of early childhood or parenting education teachers do not have a college degree 
compared to only 9 percent of adult education teachers. 

Even Start staff reported receiving an average of 40 hours of inservice training per year. 
Sessions most frequently focus on curriculum and instruction in the core service areas. 
Staff attended state, regional, and national conferences on family literacy, early childhood 
education, adult education, vocational education, and bilingual education. 



Participation in Even Start 

The great majority of Even Start families participate in early childhood education, 
parenting education, and adult education services. The proportion of families participating 
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in all three core services is much higher in mature projects than in projects in their first 
or second year of operation. 

Number of Participating Families. The number of families participating in Even Start has 
increased over time. This is due both to the addition of new projects and to improved 
efficiency among existing projects. During the 1989-90 school year, when Even Start 
began, 76 projects served about 2,500 families. By the 1992-93 school year, the 
program had grown to 340 projects which served about 20,000 families. 

Length of Participation. Although Even Start projects generally are funded for four years, 
relatively few families take part for that amount of time. Of the families that began Even 
Start in 1989-90, 53 percent participated only in that first year, 24 percent participated 
in both the first and second program years, 13 percent participated in the first three 
program years, and 10 percent participated in all four years. 

Reasons for Leaving. There are many reasons that families end their participation in Even 
Start; some are indicators of program success, while others signal problem areas. 
Seventy percent of the families that left the program indicated a reason for doing so. 
Based on families that reported a reason for leaving, completion of the planned 
educational program or meeting personal objectives was listed by 24 percent of families 
that exited Even Start. Moving out of Even Start's catchment area was listed for 24 
percent of families that exited. Thirteen percent of the families left because of a general 
lack of interest in the program and a subsequent refusal to participate. Another 12 
percent reported that a family crisis prevented them from participating. Ten percent left 
because they became ineligible due to a change in the family situation, i.e., there was no 
longer an eligible child or adult in the family. Five percent gave a variety of reasons 
including medical problems, work conflicts, pregnancy, scheduling conflicts, child care 
problems, and a lack of transportation. 

Participation in Core Services. The Department of Education requires that all Even Start 
families participate in the three core service areas during their time in the program. 

• Almost all families had a child that participated in early childhood 
education during each year of the study: 90 percent in 1 989-90, 97 
percent in 1990-91, and 98 percent in 1991-92 and 1992-93. 

• Participation rates for parenting education were a little lower: 88 
percent in 1 989-90, 94 percent in 1 990-91 , 93 percent in 1 991-92, 
and 95 percent in 1992-93. 

• At the beginning of Even Start, participation rates were low for adult 
education; 54 percent in 1989-90. This was due, in part, to 
misunderstandings about the definition of legitimate adult education 
services as well as problems convincing adults to participate on a 
regular basis. The Department of Education and local projects 
worked hard to increase these rates to 80 percent in 1990-91, 90 
percent in 1991-92, and 91 percent in 1992-93. 
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• The percentage of families participating in all three core services 
increased steadily over the four years of the study, from 46 percent 
to 75 percent to 84 percent to 86 percent. 

Amount of Core Services Received. The typical Even Start family received core services 
in seven different months during their period of enrollment in Even Start. 

• The total amount of service received by the average family is 107 
hours for adult education (median of 41 hours), 58 hours for 
parenting education (median of 29 hours), and 232 hours for early 
childhood education (median of 102 hours). There is great variation 
in these numbers across projects. 

• On a monthly basis, The average Even Start family received 13.5 
hours of adult education (median of 9 hours), 6.5 hours of parenting 
education (median of 4 hours), and 26 hours of early childhood 
education (median of 14 hours). 

There is a clear relationship between amount of early childhood education and age of 
child. The average Even Start child less than one year of age received 15.5 hours per 
month. One-year-olds received 14.3 hours per month, two-year-olds received 20.0 hours 
per month, three-year-olds received 29.2 hours per month, and four-year-olds received 
35.5 hours per month (this is about 25 percent less than the average amount of early 
chitdhood education received by Head Start children). 

Relationship Between Home-Based Services, Retention, and Amount of Service. There 
is a strong positive relationship between the amount of home-based service provided by 
a project and retention/participation in Even Start. Program retention, defined as the 
percentage of families participating in Even Start for six or more months, increases from 
about 40 percent in projects that provided no home-based services, to about 70 percent 
in projects that provided two-thirds or more of their services in the home. At the same 
time that retention is improved, hours of service are decreased in projects that emphasize 
home-based instruction. That is, the average hours of instruction per month received by 
participating families decreased as the amount of home-based instruction increased. It 
is easy to have high amounts of instruction in a center-based early childhood program 
where children participate as a group, while a home-based program which includes one- 
on-one instruction rarely will be able to provide more than one or two hours per week. 



Effects of Even Start on Services Received 

Our ability to attribute any observed changes in adults or children to Even Start as 
opposed to other factors is dependent on the extent to which Even Start has been able 
to increase families' participation in the three core service areas over what it would have 
been in the absence of Even Start. Data from this study show that Even Start has 
substantially increased participation rates in the three core service areas. 
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• We estimate that without Even Start, 30 to 40 percent of the Even 
Start population would take part in adult education. This is 
substantially less than the 91 percent participation rate achieved for 
Even Start families during the 1992-93 program year. 

• For parenting education, participation increases from 8 percent 
without Even Start to 95 percent for families in Even Start. 

• For early childhood education, participation rates are estimated to be 
about 60 percent in the absence of Even Start and 98 percent for 
families in Even Start. 

These increases in service levels indicate that Even Start projects achieved the important 
step of ensuring that program services actually were received by participating families. 



Effects of Even Start on Children 

Children in Even Start acquired school readiness skills earlier than their counterparts in a 
control group. Further, children of adults with high amounts of exposure to parenting 
education classes jained more on a vocabulary test than children of adults with less 
exposure to parenting education. 

School Readiness. The PreSchool Inventory (PSI) measures school readiness skills (e.g., 
shapes, colors, sizes) and we expect children's scores on the PSI to improve with age, 
simply as a function of maturation. Based on data collected through the NEIS (and from 
many other studies that have used the PSI), we know that children's PSI scores increase 
by about .40 items per month due to normal maturation. 

• By participating in Even Start, children's PSI scores increased at 
more than double the expected rate, by .91 items per month. 

• Data from the In-Depth Study show that Even Start children gained 
more than control group children during their early participation in 
the program. 

• However, follow-up measurements show that after 18 months 
control group children caught up with their Even Start counterparts, 
both because control group children enrolled in preschool or 
kindergarten, and because some Even Start children no longer 
participated in an Even Start early childhood program. 

• Data from the NEIS show a positive relationship between amount of 
participation in early childhood education and gains on the PSI. 

When they were three or four years of age, Eiven Start children were more likely than 
control group children to participate in early childhood education programs. Because of 
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this, they showed an accelerated rate of learning on the PSI, suggesting that they learned 
basic concepts and precursors of kindergarten skills at an earlier age than they would 
have in the absence of the program. However, as control group children reached four and 
five years of age they, too, enrolled in preschool or reached public school age. At that 
point, they learned many Of the same skills that Even Start children had learned at a 
younger age. What is not known is the long-term effect of the early boost in learning that 
was experienced by Even Start children. It is possible that since Even Start children 
learned readiness skills prior to entering the public schools, they may have progressed to 
other skills during kindergarten. 

Receptive Vocabulary. The Peabody Picture Vocabulary Test (PPVT) measures hearing 
(receptive) vocabulary. Unlike the PSI, the PPVT has national norms. When given the 
PPVT as a pretest, Even Start children averaged 80 standard score points. This 
corresponds to the 9th percentile nationally and highlights the low verbal skills of children 
at entry to Even Start. 

When compared with national norms, data from the NEIS indicate that Even Start children 
gained more than would be expected without the program; when posttested, they scored 
at the 1 9th percentile nationally. However, the In-Depth Study shows that Even Start 
children gained no more than their control group counterparts. As was the case for the 
PSI, data from the NEIS demonstrate a positive relationship between amount of early 
childhood education service and gains on the PPVT. 

Emergent Literacy Skills. The Children's Emergent Literacy Test (CELT) was developed 
by Abt Associates for this evaluation because no existing measure of children's emergent 
literacy skills was ideally suited to the Even Start evaluation. The CELT includes items 
assessing orientation and directionality of text, recognition or letters and punctuation, and 
the purposes of reading. It was administered only in the In-Depth Study, and while Even 
Start children gained a significant amount, there was no program impact because control 
group children also gained on this measure. 

Correlates of Child Effects. Greater exposure to early childhood education is associated 
with greater gains for children on the PSl and PPVT. Just as important, and more relevant 
to the special focus of Even Start, there is a statistically significant positive relationship 
between the amount of time that parents spend in parenting education and their childrens' 
PPVT scores. In particular, PPVT scores are expected to increase by 1 .1 points for each 
100 hours that a child participates in early childhood education and by an additional 1 .4 
points for each 100 hours that a parent participates in parenting education. Although this 
finding is derived from a correlational (rather than experimental) analysis, it suggests that 
the Even Start model has added benefits for children beyond the expected benefits that 
are generally derived from a traditional early childhood education program. 
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Effects of Even Start on Parent Literacy 



Even Start helped many adults acquire a GED. While adults who participate in Even Start 
achieved positive gains in functional literacy in reading, adults in a control group also 
achieved some gains. 

GED Attainment. Many Even Start projects promote attainment of a high school diploma 
or a high school equivalency diploma such as the GED certificate, as do most adult 
education programs. The rationale for this goal is based upon a belief that a diploma or 
GED increases an adult's chances to find employment, provides opportunities for higher 
wages, and improves the quality of life through enhanced self-esteem. However, GED 
attainment is not an immediate or appropriate goal for all participants, nor is it emphasized 
by all projects. 

Even Start has a strong, positive effect on GED attainment. Data from both the In-Depth 
Study and the NEIS lead to the same conclusion: Even Start has led to a substantial 
increase in the percentage of adults attaining a GED. 

• Across all Even Start projects, 8.3 percent of all adults who entered 
Even Start without a GED or diploma achieved one while 
participating in adult education services. 

• Data from the In-Depth Study show that 22.4 percent of Even Start 
adults attained a GED compared to 5.7 percent of adults in control 
group families. 

Functional Literacy in Reading. Even Start projects may have been effective in improving 
the functional literacy of participating adults but the results from this evaluation are not 
conclusive. 

• Data from the NEIS show that adults who participate in Even Start 
achieve positive gains on the CASAS reading survey, gains which 
are comparable or greater in size than those observed in other 
studies of adult education programs. 

• However, data from the In-Depth Study show that the gains of Even 
Start adults are not significantly greater than the gains achieved by 
a randomly assigned control group, so we cannot conclude that 
Even Start has had a positive impact in this area. 

• Even Start adults who spent large amounts of time in adult 
education had greater CASAS gains than adults who spent small 
amounts of time in adult education. This is the same relationship 
that was seen between participation in early childhood education 
and child-level gains on the PSI and PPVT. 
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Reading/Writing in the Home. No measurable effects were detected over an 18-month 
period on the extent to which parents read and write as part of their daily lives at home. 

Effects of Even Start on Parenting Skills 

Even Start did not measurably improve parenting skills in areas such as self-esteem, 
depression, or positive parent/child interactions. This could be due to a weakness in our 
ability to measure these areas, to generally positive parent responses and high self-ratings 
at pretest, or to a lack of program effectiveness. 

Parents' Personal Skills. Parents' personal sense of well-being, as measured by a sense 
of mastery over their lives and a lack of depression, has been cited in the research 
literature as being related to the nature and quality of parent-child and family 
relationships. Data from the In-Depth Study did not reveal any effect of Even Start on 
these variables. The prevalence of mothers with depressive symptoms as measured by 
the CES-D scale was high among this group of families (46 percent of Even Start mothers 
were classified as having high levels of depressive symptoms at the time of the pretest), 
although this is no higher than what has been reported in other studies of low-income 
women with small children. 

Home Learning Environment. Several scales describing different aspects of the home 
learning environment were included both in the In-Depth Study and in the NEIS (e.g., 
measures of learning activities in the home, teaching child at home, talking with child at 
home, family rules, activities outside home, and the Parent As A Teacher scale). While 
all variables showed small but significant positive changes on the NEIS, the In-Depth 
Study found a positive program effect only on one scale: the number of reading materials 
in the home. This scale measures the number of different types of reading materials 
found in the home, e.g... books, magazines, newspapers. This finding is consistent with 
the observation that many Even Start projects make a concerted effort to increase the 
number of reading materials in participants' homes, either through loans or donations. 

Parent-Child Reading Task. The In-Depth Study included a newly-developed measure of 
parent-child reading interactions. The parent was asked to read a simple book to her child 
while a trained observer used a pre-coded rating form to record five major aspects of 
parent-child interactions: parent describes book to child, parent questions or responds to 
child, child responds, parent's general affect, and child's general affect. There was no 
effect of Even Start on any of the five areas of observation. 

Parental Expectations. In both the In-Depth Study and the NEIS, parents were asked 
questions about how well they expected their child to do in high school and the likelihood 
that their child will graduate from high school. As is the case for such questions in most 
studies, parents generally had high expectations for their children at the pretest. Even 
Start significantly increased parental expectations as assessed by the NEIS but not the 
In-Depth Study. 
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Effects of Even Start on Families 

The In-Depth Study included several measures of the effects of Even Start on participating 
families including their perceived social support, adequacy of financial resources, income 
level and sources, and employment status. The NEIS instruments had questions about 
income and employment. There were no significant program effects on any of these 
measures. This is not surprising -- these areas are very difficult to change because they 
are affected by circumstances beyond the control of the program, such as the local 
economy and the availability of jobs. It is likely that longer interventions and other 
positive short-term impacts will be required in order to achieve significant gains on these 
measures. 

While there are no effects for families on these quantifiable variables, Even Start 
participants described a number of qualitative changes in their lives and the lives of their 
children. Listening to the personal stories of program participants during focus groups 
conducted in ten projects, it became apparent that parents saw positive changes in their 
attitudes and skills. Parents described themselves as moving toward their goals of an 
educational certificate, getting a jcb and being a better parent. Given the entry status of 
Even Start parents, it is reasonable that we do not see short-term changes in the distal 
outcomes of increased employment and income. 



Cost of Even Start 

The federal cost per Even Start family declined over the life of the program, from $5,894 
in 1989-90 to $3,669 in 1990-91, and again to $2,503 in 1991-92. This is due to 
increases in the number of families served each year, indicating that over time, projects 
have matured and become more efficient. Even Start projects also obtain substantial 
resources (e.g., matching funds, in-kind contributions, and the value of referred services), 
in addition to their federal Even Start funds, in order to deliver appropriate services to 
participating families. 

• In 1991-92, the average of $2,503 in federal funding per family was 
augmented by an average of $1 ,352 in other resources to arrive at 
total resources of $3,855 per Even Start family. Thus, federal Even 
Start funds comprise 65 percent of the total resources used per 
family and other funds comprise 35 percent. 

• Almost two-thirds (64 percent) of all resources were spent on the 
direct provision of services: 31 percent for early childhood 
education, 1 5 percent, for adult education, 9 percent for parenting 
education, and 9 percent for support services. 

• Funds also were spent for program administration and coordination 
(14 percent), evaluation (10 percent), case management and 
recruiting (4 percent), and for other functions (8 percent). 
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Even Start costs vary tremendously across projects. As might be expected, projects that 
serve large numbers of families do so at a lower cost per family. In particular, projects 
that serve 100 or more families do so at a federal cost of $1,659 per family, while 
projects that serve 30 or fewer families spend an average of $6,31 2 in federal Even Start 
funds per family. Also, projects that delegate responsibility for providing core services 
to cooperating agencies have a lower cost ($1,878 per family) than projects that retain 
primary responsibility for providing core services (cost of $5,775 per family). 

Implementation and Use of the Evaluation 

Implementation of the Evaluation. The national Even Start evaluation was implemented 
as a collaborative effort between staff at the U.S. Department of Education, the staff of 
the evaluation contractors, and the staf f of each Even Start project, it was hoped that 
a greater than usual level of involvement by Even Start grantees would benefit the 
evaluation through an increased investment and level of understanding on the part of 
program implementers, and through a better appreciation of programmatic issues and 
problems on the part of evaluation staff. 

Responsibilities for the national evaluation were divided such that oversight came from 
the Department of Education; design, analysis, reporting, and technical assistance as well 
as data collection for the In-Depth Study were provided by the evaluation contractors; and 
input to the design and analysis, interpretation of findings, and primary responsibility for 
data collection for the NEIS were provided by local Even Start projects. To enable 
projects to undertake their evaluation responsibilities, they were given additional grant 
funds which were used to hire a local evaluator who helped interact with the national 
evaluation, assisted with data collection, and sometimes designed and conducted their 
own local evaluation activities to supplement the national evaluation. 

Staff from each Even Start project were involved in an early design meeting where 
feedback was obtained on drafts of the data collection forms, and where a working group 
of Even Start project directors and local evaluators was established. Subsequent drafts 
of data collection forms were shared first with the working grouo, and then with all 
project directors and local evaluators. Training sessions in administering the data 
collection instruments were held for all Even Start projects; additional feedback on forms 
and on data collection problems was provided through formal meetings, mailings, and 
telephone-based technical assistance. 

This approach was seen as experimental, and at the start of the evaluation it was unclear 
whether local project staff would be willing and able to undertake a high-quality data 
collection effort. After four years of experience, it is possible to draw the following 
conclusions about the implementation of the evaluation. 

• Many Even Start projects were comfortable with their role as data 
collectors and were able to turn in high-quality data according to the 
time schedule set forth for the evaluation. Others needed 
substantial assistance in complying with the requirements of the 



Even Start 



251 



Summary and Conclusions 



evaluation, and several projects were not able to provide data 
according to the hoped-for timetable. 

• Most Even Start projects availed themselves of the evaluation 
technical assistance provided by the evaluation contractor, and 
would have preferred more. 

• Data collection was done by local staff with a wide range of 
backgrounds. In some cases, local evaluators did the data 
collection. In others, project staff (e.g., case managers, early 
childhood teachers) conducted the data collection. 

• The arrangement to have data collected by local project staff has 
the drawbacks of potential bias, lack of attention to important 
details (e.g., matching family identification numbers over time), and 
work-load conflicts between evaluation and programmatic activities. 
The training and technical assistance provided by the national 
contractor worked to ameliorate these problems. On balance, we 
believe that the reliability of the data collection conducted under this 
model is somewhat lower than what would be attained by trained 
researchers, but that the response rates obtained by local staff are 
probably higher than the response rates that would be obtained by 
outside staff. 

• There was great variability in the interest and ability of Even Start 
grantees to collect data using a microcomputer-based system. This 
system worked well for grantees with access to up-to-date 
equipment and staff who understood the basics of computer 
systems. Other grantees struggled and were frustrated with this 
approach, leading to delays in the submission of data. Still others 
were not able to use a computer-based system, and submitted data 
on paper forms. 

Use of the Evaluation. The evaluation was an integral part of Even Start's development. 
The Department of Education's requirement to collect a standard set of data on the 
characteristics of program participants, the operations of each project, and the effects of 
Even Start led to implementation of an information system that ensured that grantees had 
common definitions for key programmatic activities. For example, critical definitions such 
as "Who counts as a program participant?" and "What counts as a core service?" were 
clarified by the neod to provide data for the evaluation. 

Descriptive data from the evaluation identified achievements and confirmed that Even 
Start projects served the intended population and that projects were successful in 
establishing cooperative relationships with local organizations. 

Data from the first year 0 f the evaluation plus ongoing monitoring conducted by the 
Department of Education discovered a need to revise the program's eligibility requirements 
so that a family was not terminated from participation when a parent acquired a CED or 
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a child reached age eight. The legislation was changed so that a family may participate 
in Even Start until all family members become ineligible. 

Additionally, the evaluation discovered that a significant number of Even Start projects 
had difficulty persuading adults to participate fully in adult education programs. This led 
the Department of Education to issue non-regulatory guidance as well as provide technical 
assistance in implementing adult education services for the range of skill levels among 
Even Start families. Participation rates in the adult education component of Even Start 
did rise over time, likely as a result of the Department's efforts and increased maturity on 
the part of projects. 

Finally, data from the NEIS were used by individual local projects in preparing local 
evaluations, and proved to be instrumental in helping some projects gain approval by the 
Department of Education's Program Effectiveness Panel. 



Conclusions 

We conclude this report by returning to a subset of the research questions that guided the 
evaluation and briefly summarizing the answers to each question. 

What are Even Start's "best practices"? What types of projects or program elements 
work best under what conditions? 

While evaluating "best practices" directly was beyond the scope of this project, some 
relevant conclusions can be drawn: 

• Program retention is increased in projects that provide home-based 
services. 

* More time spent receiving services is associated with greater gains 
on literacy tests for adults and children. 

Providing home-based services is a good way to increase retention. There is a strong, 
positive relationship between the amount of home-based services and the length of 
program participation. In all three core services, the proportion of families who stay in 
the program more than six months goes up from 40 percent in projects with minimal 
home-based sewices to 70 percent in projects where a majority of services are provided 
in the home. 

A greater amount of exposure to Even Start's core services (i.e. larger amounts of time 
spent receiving core services) appears to have a positive impact on literacy outcomes for 
children and adults. In other words, more time spent in early childhood education is 
associated with greater school readiness and vocabulary gains for children, and more time 
spent in adult education is associated with greater functional literacy gains for adults. 
The findings about home-based services and amount of service exposure work against 
each other, since it is difficult to deliver large amounts of instructional service in a home- 
based setting. 
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Evaluating "best practices" directly was beyond the scope of this evaluation. Anecdotal 
reports from the In-Depth Study site visits identified many innovative instructional 
strategies that were used with parents and children which could benefit from additional 
research. Examples include: 

• Using computers in adult education classrooms to write newsletters 
sharing personal experiences and program activities and to institute 
a pen pal program with other Even Start projects. 

• Incorporating newspapers into parenting and adult education classes 
through daily discussions of current events, distribution of free 
Spanish-language papers, and written homework about newspaper 
articles. 

• Integrating adult and early child education by having parents write 
stories for their children and make books to read at home. 

• Enhancing early childhood classrooms with signs and symbols to 
foster emergent literacy skills. 

• Hiring social workers to provide parenting education and staff 
training. 

• Using a team approach to staffing to facilitate integration of adult 
and child curricula and instruction in the classroom and during home 
visits. 



How does the program compare to alternative programs addressing the same problem? 
Is it more effective? How do the costs compare? 

This question was not addressed in the current evaluation. However, how Even Start 
compares to alternative programs in cost and effectiveness is a question that has been 
asked by researchers and policymakers over the course of the evaluation, and merits 
further study. 



What is the program's impact on its target population and service delivery structure? 

Even Start is not large enough to make an impact on its entire target population. 
However, it has had a broad impact on individual families and many local service 
communities. 

Some limited, anecdotal information about Even Start's impact on the local service 
delivery structure was collected as part of the In-Depth Study. In that component of the 
evaluation, project directors described how some school districts had incorporated aspects 
of Even Start, such as home visits or parent involvement, into the district's philosophy 
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and approach. In order to gather more systematic data about community impacts, a 
separate study focusing on that topic would be necessary. 

Even Start is too small a program to make a major impact on its entire target population. 
In spite of this caveat it is important to take note of several areas which indicate that 
Even Start has had a broad impact. First, Even Start funding has grown substantially 
each year since 1989. In a time of fiscal austerity, this is a clear and visible signal that 
Even Start proponents have been able to make a convincing case for the benefits of their 
program. Second, Even Start projects now exist in every state in the nation -- a further 
indication of Even Start's widespread support. Third, anecdotal evidence shows that local 
Even Start projects have generated a wealth of valuable and practical ideas about 
implementing family literacy programs - ideas which have helped other program 
developers. 

How well does the basic Even Start model work? 

Even Start has been well-implemented over a period of years, leading to short-term 
improvements in the lives of many adults and their children. Based on the findings from 
this evaluation, we can draw several conclusions about the viability of the basic Even 
Start model: 

• The model takes multiple years to fully implement. While 
implementation is not easy, almost all projects are able to offer all 
three core services and multiple support services, to offer some 
home-based services, and some services to parents and children 
together, and to make appropriate collaborative arrangements with 
local service providers. 

• As they matured in their third and fourth years, Even Start projects 
were able to use the same amount of resources to recruit and serve 
about twice as many families as they served in their first year. 

• The total amount (i.e., number of hours of participation) of core 
instructional services received by participating families is important. 
A larger amount of participation in core services is associated with 
larger test gains for adults and children. 

• Participation in adult education services led many Even Start adults 
to attain a GED certificate. 

• Participation in early childhood education led Even Start children to 
attain school readiness skills sooner than they otherwise would 
have. 

• Greater parents' participation in parenting education is related to 
increases in their children's vocabularies. 
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Even Start did not produce any measurable effects on parenting 
skills, family resources, income, or employment. 

Program retention is increased in projects that provide larger 
amounts of home-based services. 



The current evaluation has identified areas where Even Start has had short-term effects 
on children and parents. However, in order to evaluate long-term effects on program 
participants, a longitudinal study is required. For example, the long-term effect of Even 
Start on children's school behaviors and cognitive performance is a critical element in 
assessing Even Start's impact. To investigate this, a study is needed that follows Even 
Start children into the public schools and collects information from teachers, parents, and 
students. 
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